imagenet classification with deep convolutional neural networks presenter: weicong chen

11

ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Upload: lee-flynn

Post on 18-Jan-2018

222 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

DESCRIPTION

ImageNet LSVRC-2010 Contest 1.2 million high-resolution images 1,000 different classes 50,000 validation images, 150,000 testing images Top-1 error 47.1% best in contest, 45.7% best published Top-5 errors 28.2% best in contest, 25.7% best published

TRANSCRIPT

Page 1: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

ImageNet Classification with Deep Convolutional Neural

Networks

Presenter: Weicong Chen

Page 2: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Deep Convolutional Neural Networks• Led by Geoffrey Hinton, University of Toronto

• Published in 2013

• Based on the datasets from ImageNet LSVRC-2010 Contest

• Using graphic cards to train the neural network

Page 3: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

ImageNet LSVRC-2010 Contest • 1.2 million high-resolution images

• 1,000 different classes

• 50,000 validation images, 150,000 testing images

• Top-1 error 47.1% best in contest, 45.7% best published

• Top-5 errors 28.2% best in contest, 25.7% best published

Page 4: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Convolutional Neural Networks (CNN)

Page 5: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Architecture

Page 6: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Novel/Unusual features in architecture• ReLU (Rectified Linear Units ) Nonlinearity

Standard way: f(x) = tanh(x) or f(x) = (1 + e-x)-1 (logistic function)ReLU: f(x) = max(0, x)6 times faster than hyperbolic function

• Multiple GPUsLarger memoryParallel computingControl of communication

Page 7: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Overfitting• Occurs when a statistical model describes random error or noise

instead of the underlying relationship

• Exaggerate minor fluctuations in the data

• Will generally have poor predictive performance

Page 8: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Reducing overfitting• Data Augmentation1.Image translation and horizontal reflection

Randomly extracting patchesFour corner and one center patches with reflection for testing

2.Altering the intensities of the RGB channels in training imagesApproximately captures an important property of natural images reduces the top-1 error rate by over 1%

Page 9: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Reducing Overfitting• Dropout

Zero the output of each hidden neuron with probability 0.5.

No longer contribute to forward pass and backward propagation

Neural network samples a different architecture every time

Reduce complex co-adaptations of neurons

Used in two fully-connected layers

Page 10: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Result• Two NVIDIA GTX 580 3GB GPUs

• 6 days of training

• 90 cycles

• 60 million parameters

• 37.5% top-1 error (45.7% second best)

• 17% top-5 error (28.2% second best)

Page 11: ImageNet Classification with Deep Convolutional Neural Networks Presenter: Weicong Chen

Questions?

ImageNet Classification with Deep Convolutional Neural ... neural network, which has 60 million parameters and 650,000 neurons, consists of ﬁve convolutional layers, some of which

Convolutional Neural Networks arXiv:1603.05279v1 … · XNOR-Net: ImageNet Classiﬁcation Using Binary Convolutional Neural Networks Mohammad Rastegari y, Vicente Ordonez , Joseph

Convolutional Neural Networkslvelho.impa.br/ip16/proj/slides/ImageNet_DeepCNN.pdf · Convolutional Neural Networks 7 Motivation It was one of the largest CNN trained with the ImageNet

Convolutional Neural Networks - Virginia Techjbhuang/teaching/ece5554-4554/fa17/... · Convolutional Neural Networks Computer Vision ... ImageNet Classification with Deep Convolutional

ImageNet Classiﬁcation with Deep Convolutional Neural Networks

ImageNet Classification with Deep Convolutional Neural ... et al DNN ImageNet 2012.pdfneural networks to date on the subsets of ImageNet used in the ILSVRC-2010 and ILSVRC-2012

ImageNet Classification with Deep Convolutional Neural ...rogerioferis.com/.../presentations/GuangnanAndMajaDeepLearning.pdf · ImageNet Classification with Deep Convolutional Neural

Convolutional Neural Networks II - UMIACS · •ImageNet Large Scale Visual Recognition Challenge ... ConvNet is a sequence of Convolutional Layers,

XNOR-Net: ImageNet Classiﬁcation Using Binary ... · XNOR-Net: ImageNet Classiﬁcation Using Binary Convolutional Neural Networks 3 To the best of our knowledge this paper is the

Optimizing Memory Efficiency for Deep Convolutional Neural ... · The success of the deep Convolutional Neural Network (CNN), Alex-Net [12], in the 2012 ImageNet recognition competition

Tiny ImageNet Classiﬁcation with Convolutional …cs231n.stanford.edu/reports/2015/pdfs/leonyao_final.pdfTiny ImageNet Classiﬁcation with Convolutional Neural Networks Leon Yao,

ImageNet Classification with Deep Convolutional Neural ...fritz/absps/imagenet.pdf · ImageNet Classiﬁcation with Deep Convolutional Neural Networks Alex Krizhevsky University of

Deep Convolutional Neural Networks for Tiny ImageNet

Scaling SGD Batch Size to 32K for ImageNet Trainingyouyang/publications/batch32k_slide.pdfAlex Krizhevsky, One weird trick for parallelizing convolutional neural networks, 2014 (Google

ImageNet Classification with Deep Convolutional Neural ...web.cs.ucdavis.edu/~yjlee/teaching/ecs289g-fall2016/yuanzhe.pdf · ImageNet Classification with Deep Convolutional Neural

Convolutional Neural Networks - LMU Munich · 2018-11-13 · Convolutional Neural Networks Presenter: Dr. Denis Krompaß Siemens Corporate Technology – Machine Intelligence Group

ImageNet Classiﬁcation with Deep Convolutional Neural Networks de faculteit/Afdelinge… · ImageNet is a dataset of over 15 million labeled high-resolution images belonging to

XNOR-Net: ImageNet Classiﬁcation Using Binary ...ai2-website.s3.amazonaws.com/publications/XNOR-Net.pdf · XNOR-Net: ImageNet Classiﬁcation Using Binary Convolutional Neural Networks

ImageNet Classiﬁcation with Deep Convolutional Neural …hinton/absps/imagenet.pdf · ImageNet Classiﬁcation with Deep Convolutional Neural Networks Alex Krizhevsky ... and 17.0%

AutoEncoder Squeezed Convolutional Variationalpeople.cs.pitt.edu/~mosse/courses/cs3720/Squeezed... · 2018-04-17 · Squeezed Convolutional Variational AutoEncoder Presenter: Keren

ImageNet Classiﬁcation with Deep Convolutional Neural Networksvision.cse.psu.edu/people/chrisF/deep-learning/... · In all, there are roughly 1.2 million training images, 50,000

ImageNet Classiﬁcation with Deep Convolutional Neural Networks › ~hinton › absps › imagenet.pdf · ImageNet Classiﬁcation with Deep Convolutional Neural Networks Alex Krizhevsky

ImageNet Classification with Deep Convolutional Neural ... · Convolutional neural networks Output Hidden Data Here's a one-dimensional convolutional neural network Each hidden neuron

ImageNet Classification with Deep Convolutional Neural Networksstat.snu.ac.kr/idea/seminar/20170504/Alexnet.pdf · 2017-05-19 · ImageNet Classi cation with Deep Convolutional Neural

ImageNet Classiﬁcation Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classiﬁcation Using Binary Convolutional Neural Networks

Do ImageNet Classiﬁers Generalize to ImageNet?proceedings.mlr.press/v97/recht19a/recht19a.pdf · 2021. 1. 14. · Do ImageNet Classiﬁers Generalize to ImageNet? Figure 1. Model

ImageNet Classification with Deep Convolutional Neural Networks

on Classification Convolutional - York University · Review on ImageNet Classification with Deep Convolutional Neural ... 70K 28*28 grey‐level handwritten digits for 10 ... CUDA

ImageNet Classification with Deep Convolutional …• A deep convolutional neural network is trained to classify the 1.2 million ImageNet images into 1000 different classes. • The

Global Optimality in Matrix and Tensor Factorization, Deep ...€¦ · Factorization, Deep Learning & Beyond ... "Imagenet classification with deep convolutional neural networks."NIPS,

Lab3 EM & GCN - GitHub Pages · Lab3 EM & GCN Chen Houshuang [email protected]. contents GCN EM. Graph Convolutional Networks. Traditional Deep Learning •CNN ImageNet Grid

Part 2: Convolutional Neural Networks · 2020-03-22 · Part 2: Convolutional Neural Networks Dr. Adam Teman EnICS Labs, Bar-Ilan University Lecture Series on ... •“ImageNet classification

imagenet classification

Medical ImageNet

ImageNet Classiﬁcation with Deep Convolutional Neural …stanford.edu/class/cs231m/references/alexnet.pdfOur network contains a number of new and unusual features which improve its