neural networks, deep learning and applications
TRANSCRIPT
![Page 1: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/1.jpg)
Universidad del Valle de Guatemala
Agosto 07, 2018
Neural Networks, Deep Learning
and Applications
Alan Gerardo Reyes Figueroa
Centro de Investigación en Matemáticas
CIMAT
![Page 2: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/2.jpg)
Inteligencia ArtificialG. Kasparov vs. DeepBlue
(1997)
Lee Sedol vs. AlphaGo
(2016)
![Page 3: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/3.jpg)
Inteligencia Artificial
Self-driving cars.
![Page 4: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/4.jpg)
Inteligencia Artificial
• ¿En qué consiste?
• Otros términos:
- big data
- machine learning
- deep learning
- analytics
- cognitive systems …
![Page 5: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/5.jpg)
Inteligencia Artificial
• Learning theory
• Psychology
![Page 6: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/6.jpg)
Inteligencia Artificial
![Page 7: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/7.jpg)
Inteligencia Artificial
![Page 8: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/8.jpg)
Redes Neuronales
• Algoritmos bioinspirados.
• Idealizan cómo funcionan las conexiones y transmisiones dentro del cerebro biológico.
![Page 9: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/9.jpg)
Redes Neuronales
![Page 10: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/10.jpg)
Redes Neuronales Artificiales
![Page 11: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/11.jpg)
Redes Neuronales Artificiales
linear component
non-linear component
activation
function
![Page 12: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/12.jpg)
Redes Neuronales Artificiales
Inputs
“Hidden” layers
Outputs
![Page 13: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/13.jpg)
Desarrollo Redes Neuronales
• Algoritmos bioinspirados.
• Idealizan cómo funcionan las conexiones y transmisiones dentro del cerebro biológico.
![Page 14: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/14.jpg)
McCulloch-Pitts Model (1940’s)
![Page 15: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/15.jpg)
McCulloch-Pitts Model (1940’s)
![Page 16: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/16.jpg)
Rosenblatt’s Model (1960’s)
![Page 17: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/17.jpg)
Rosenblatt’s Model (1960’s)
![Page 18: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/18.jpg)
Rosenblatt’s Model (1960’s)
![Page 19: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/19.jpg)
Gradient Descent
![Page 20: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/20.jpg)
Rosenblatt’s Model (1960’s)
![Page 21: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/21.jpg)
Widrow-Hoff Model (1960’s)
![Page 22: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/22.jpg)
The Golden Age (1960’s)
• Curva de Gartner para una tecnología
![Page 23: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/23.jpg)
Dark Ages (1970’s)
• Minsky y Papert (1969) publican Perceptrons: an introduction to computational geometry.
• Crítica y limitaciones al modelo perceptrón.
• “Invierno” de la inteligencia artificial.
XOR
function
![Page 24: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/24.jpg)
Segunda Era (1980’s)
• Rumelhart, Hinton y Williams (1986)
Nature: Learning representations by
back-propagating errors.
• Solución al problema de la separabilidad.
XOR
function
![Page 25: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/25.jpg)
Segunda Era (1980’s)
• Redes multicapa.
• Algoritmo de back-propagation.
![Page 26: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/26.jpg)
Segunda Era (1980’s)
• K. Fukushima (79’)
NeoCognitron.
Trabajos en redes multicapa.
• Teoría matemática de redes neuronales.
• G. Cybenko (1989).
Teorema de representación universal.
![Page 27: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/27.jpg)
Aproximadores universales
![Page 28: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/28.jpg)
Teorema de Cybenko
![Page 29: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/29.jpg)
Teorema de Aproximación
• Si la función de activación ϕ satisface “ciertas condiciones”, las redes multi-capason capaces de aproximar cualquier función.
(X compacto)
• Las redes neuronales se pueden adaptar a cualquier conjunto (finito) de datos.
![Page 30: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/30.jpg)
Teorema de Aproximación
• Composición de transformaciones lineales:
linear
![Page 31: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/31.jpg)
Geometría de las redes neuronales
![Page 32: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/32.jpg)
Ejemplo: Kernel Trick
![Page 33: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/33.jpg)
Ejemplo• Geometría de las redes neuronales.
![Page 34: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/34.jpg)
Manifold Hypothesis
![Page 35: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/35.jpg)
Second winter (1990’s)
• Financiamiento sector militar.
• Sobre-expectativa en proyectos.
- autos autónomos
- máquinas inteligentes
• Reducción en el apoyo a proyectos del área“Inteligencia Artificial”.
• Cambio de nombre: informatics, machine learning,
analytics, knowledge-based systems, business rules management, cognitive systems, intelligent systems,intelligent agents or computational intelligence
![Page 36: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/36.jpg)
Siglo XXI (2000 – hoy)
• Era de la información: redes cada vez mayores.
• Industria de cómputo (gaming) y de cómputo distribuido
• Inversión de grandes compañías (Nvidia, Google, Facebook, Amazon, Baidu, …)
![Page 37: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/37.jpg)
Estado actual de la IA
![Page 38: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/38.jpg)
Deep Learning (2012 - )
• ImageNET Challenge (2010)
- clasificar 14 millones de imágenes
• AlexNet (2012)
- breakthrough de las redes neuronales
![Page 39: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/39.jpg)
Deep Learning (2012 - )
![Page 40: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/40.jpg)
Deep Learning (2012 - )
• Perceptron
• Two-layer network
• GoogleNet (2014)
![Page 41: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/41.jpg)
Deep Learning (2012 - )Topologías especializadas (bloques básicos)
• Fully-connected
- clasificación
• Convolutional
- filtrado
- visión
• Recurrent
- evolución
- lenguaje
![Page 42: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/42.jpg)
Era pre – Deep Learning
![Page 43: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/43.jpg)
Era post – Deep Learning
![Page 44: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/44.jpg)
Aplicaciones
![Page 45: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/45.jpg)
Aplicaciones
• Clasificación
• Predicción
• Generación
• Visión Computacional
• Procesamiento de Lenguaje
• Diagnóstico Médico
• Prevención de desastres
• Robótica
![Page 46: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/46.jpg)
Computer Vision
Image Segmentation
![Page 47: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/47.jpg)
Computer Vision
Object Detection
![Page 48: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/48.jpg)
Self-Driving Cars
![Page 49: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/49.jpg)
Noise Removal
Speckle noise removal in fringe patterns.
![Page 50: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/50.jpg)
Noise Removal
Noise removal from images using Deep neural networks
![Page 51: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/51.jpg)
Restoration
Automated colorization of images.
![Page 52: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/52.jpg)
Restoration
Automated colorization of films.
![Page 53: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/53.jpg)
Restoration
Inpainting and art restoration (Gent Altarpiece).
![Page 54: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/54.jpg)
Super-Resolution
![Page 55: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/55.jpg)
Sistemas de seguridad
![Page 56: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/56.jpg)
Healthcare
Assisted Diagnosis
![Page 57: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/57.jpg)
Automated Diagnosis
Early detection of Health Anomalies
![Page 58: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/58.jpg)
Automated Diagnosis
Retinal vessel segmentation using deep learning
![Page 59: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/59.jpg)
Automated Diagnosis
![Page 60: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/60.jpg)
Remote Sensing- Clasificación
del uso del
suelo
- Detección de
área en riesgo
Incendios
Deslaves
- Prevención de
desastres
![Page 61: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/61.jpg)
Applications in Physics
CERN Tracking
Challenge
Solution of PDE’s
![Page 62: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/62.jpg)
Natural Language Processing
- Traducción
- Transliteración audio-texto
- Clasificación de contenidos
- Asistentes virtuales de voz
![Page 63: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/63.jpg)
Robótica
![Page 64: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/64.jpg)
Generación de Arte
![Page 65: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/65.jpg)
Art style transfer
![Page 66: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/66.jpg)
Art style transfer
![Page 67: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/67.jpg)
Art style transfer
![Page 68: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/68.jpg)
Automated art generation
- Generación de textos literarios, poemas, …
- Transliteración música y partituras
- Generación de pinturas
![Page 69: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/69.jpg)
Futuro de la IA
![Page 70: Neural Networks, Deep Learning and Applications](https://reader034.vdocuments.us/reader034/viewer/2022050306/626f403bc1ec8a4b366171d4/html5/thumbnails/70.jpg)
Gracias!