a tour of several popular tensorflow models

A TOUR OF SEVERAL POPULAR TENSORFLOW

MODELS

by Jason Toy

CHAR-RNN

“I'm not going anywhere. I will bring the poorly educated

back bigger and better. It's an incredible movement. ”

“We're losing companies, the economy. We are going to

save it. We're going to bring the party. Let's Make America

Great Again”

“I want to thank the volunteers. They've been unbelievable,

they work like endlessly, you know, they don't want to die.

My leadership is good”

CHAR-RNN

character level language modeling W-O-R-D, not WORD

good for NLP

original implementation from karpathy in python:

https://github.com/karpathy/char-rnn

dozens of implementations, including several in tensorflow

CHAR-RNN

1vanilla (image classification) 2 sequence output (image ->

text) 3 sequence input ( sentiment analysis)

4 seq2seq (machine translation) 5 synced seq2seq (video

classification)

CHAR-RNN

RNN - recurrent because they perform the same task for

every element of a sequence; typically 2-3 layers

LSTM - long short term memory

similar, state is calculated differently

CHAR-RNN DATA

input is raw, large bodies of text. 1 MB+ minimum.

modeling any kind of sequential data

people have tried: linux source code,shakespeare, trump

speeches, obama speeches,MIDI music, chinese, and more

easy to find data to test with

ideas for data you would like to try it on?

CHAR-RNN APPLICATIONS

chat bots

new “works of arts”

music generation

foundation for many other networks that involve text or

images

CHAR-RNN MORE INFO

TF version: https://github.com/somaticio/char-rnn-tensorflow

awesome explanation: http://karpathy.github.io/2015/05/21/rnn-

effectiveness/

seq2seq: arbitrary length input sequences that output arbitrary

length outputs

live version to play with:

http://www.somatic.io/models/WZmmBjZ9

tensorflow tutorial:

https://www.tensorflow.org/versions/master/tutorials/recurrent/ind

ex.html

NEURALSTYLE

Paint images in the style of any painting

A NEURAL ALGORITHM OF ARTISTIC STYLE

paper: http://arxiv.org/abs/1508.06576

The key finding of this paper is that the representations of

content and style in the CNNs are separable.

CNNs - convolutional Neural Network

high layers in the network act as the content of the image

style computed from multiple layers’ filter responses

NEURALSTYLE - OVERVIEW

original version written in torch/lua:

https://github.com/jcjohnson/neural-style

tensorflow version: https://github.com/anishathalye/neural-

style

online test model: http://www.somatic.io/models/5BkaqkMR

NEURAL STYLE DATA

VGG - 16 convolutional and 5 pooling layers of the 19 layer

VGGNetwork. it won imagenet in 2014

older network for object classification, not considered state

of the art

transfer learning

computation time

VGG Network

FUTURE REVISIONS

style transfer while retaining color

image analogies

NEURALTALK/SHOW AND TELL

UNDERSTANDABLE MESSUPS

BAD MESSUPS

SHOW AND TELL DATA

flickr30k ~150k captions on ~30k images:

http://web.engr.illinois.edu/~bplumme2/Flickr30kEntities/

after1 year, a baby has taken in approximately 260 million

images.

SHOW AND TELL

many versions,popular one by Andrew Karpathy written in

python/lua: https://github.com/karpathy/neuraltalk2

https://github.com/kelvinxu/arctic-captions

tensorflow version:

https://github.com/jazzsaxmafia/show_and_tell.tensorflow

online model to test:

http://www.somatic.io/models/qoEGanRe

paper: http://arxiv.org/abs/1411.4555

SHOW AND TELL APPLICATIONS

facebook has deployed this as image captioning software

for the blind

search engine indexing systems for movies

“storytelling” art: http://somatic.io/models/2n6g7RZQ

LINKS

char-rnn: https://github.com/somaticio/char-rnn-tensorflow

live version: http://somatic.io/models/WZmmBjZ9

tensorflow char-rnn tutorial:

https://www.tensorflow.org/versions/r0.9/tutorials/seq2seq/index.html#recurre

nt-neural-networks

neuralstyle: https://github.com/anishathalye/neural-style

live version: http://somatic.io/models/qoEGanRe

show and tell: https://github.com/jazzsaxmafia/show_and_tell.tensorflow

live version: http://somatic.io/models/2n6g7RZQ

–John Dewey

“Every great advance in science has issued from a

new audacity of imagination.”

Jason Toy

jason@somatic.io

test and use models: http://somatic.io

@jtoy

QUESTIONS?