a tour of several popular tensorflow models
TRANSCRIPT
A TOUR OF SEVERAL POPULAR TENSORFLOW
MODELS
by Jason Toy
CHAR-RNN
“I'm not going anywhere. I will bring the poorly educated
back bigger and better. It's an incredible movement. ”
“We're losing companies, the economy. We are going to
save it. We're going to bring the party. Let's Make America
Great Again”
“I want to thank the volunteers. They've been unbelievable,
they work like endlessly, you know, they don't want to die.
My leadership is good”
CHAR-RNN
character level language modeling W-O-R-D, not WORD
good for NLP
original implementation from karpathy in python:
https://github.com/karpathy/char-rnn
dozens of implementations, including several in tensorflow
CHAR-RNN
1vanilla (image classification) 2 sequence output (image ->
text) 3 sequence input ( sentiment analysis)
4 seq2seq (machine translation) 5 synced seq2seq (video
classification)
CHAR-RNN
RNN - recurrent because they perform the same task for
every element of a sequence; typically 2-3 layers
LSTM - long short term memory
similar, state is calculated differently
CHAR-RNN DATA
input is raw, large bodies of text. 1 MB+ minimum.
modeling any kind of sequential data
people have tried: linux source code,shakespeare, trump
speeches, obama speeches,MIDI music, chinese, and more
easy to find data to test with
ideas for data you would like to try it on?
CHAR-RNN APPLICATIONS
chat bots
new “works of arts”
music generation
foundation for many other networks that involve text or
images
CHAR-RNN MORE INFO
TF version: https://github.com/somaticio/char-rnn-tensorflow
awesome explanation: http://karpathy.github.io/2015/05/21/rnn-
effectiveness/
seq2seq: arbitrary length input sequences that output arbitrary
length outputs
live version to play with:
http://www.somatic.io/models/WZmmBjZ9
tensorflow tutorial:
https://www.tensorflow.org/versions/master/tutorials/recurrent/ind
ex.html
NEURALSTYLE
Paint images in the style of any painting
A NEURAL ALGORITHM OF ARTISTIC STYLE
paper: http://arxiv.org/abs/1508.06576
The key finding of this paper is that the representations of
content and style in the CNNs are separable.
CNNs - convolutional Neural Network
high layers in the network act as the content of the image
style computed from multiple layers’ filter responses
NEURALSTYLE - OVERVIEW
original version written in torch/lua:
https://github.com/jcjohnson/neural-style
tensorflow version: https://github.com/anishathalye/neural-
style
online test model: http://www.somatic.io/models/5BkaqkMR
NEURAL STYLE DATA
VGG - 16 convolutional and 5 pooling layers of the 19 layer
VGGNetwork. it won imagenet in 2014
older network for object classification, not considered state
of the art
transfer learning
computation time
VGG Network
FUTURE REVISIONS
style transfer while retaining color
image analogies
NEURALTALK/SHOW AND TELL
UNDERSTANDABLE MESSUPS
BAD MESSUPS
SHOW AND TELL DATA
flickr30k ~150k captions on ~30k images:
http://web.engr.illinois.edu/~bplumme2/Flickr30kEntities/
after1 year, a baby has taken in approximately 260 million
images.
SHOW AND TELL
many versions,popular one by Andrew Karpathy written in
python/lua: https://github.com/karpathy/neuraltalk2
https://github.com/kelvinxu/arctic-captions
tensorflow version:
https://github.com/jazzsaxmafia/show_and_tell.tensorflow
online model to test:
http://www.somatic.io/models/qoEGanRe
paper: http://arxiv.org/abs/1411.4555
SHOW AND TELL APPLICATIONS
facebook has deployed this as image captioning software
for the blind
search engine indexing systems for movies
“storytelling” art: http://somatic.io/models/2n6g7RZQ
LINKS
char-rnn: https://github.com/somaticio/char-rnn-tensorflow
live version: http://somatic.io/models/WZmmBjZ9
tensorflow char-rnn tutorial:
https://www.tensorflow.org/versions/r0.9/tutorials/seq2seq/index.html#recurre
nt-neural-networks
neuralstyle: https://github.com/anishathalye/neural-style
live version: http://somatic.io/models/qoEGanRe
show and tell: https://github.com/jazzsaxmafia/show_and_tell.tensorflow
live version: http://somatic.io/models/2n6g7RZQ
–John Dewey
“Every great advance in science has issued from a
new audacity of imagination.”
Jason Toy
test and use models: http://somatic.io
@jtoy
QUESTIONS?