deep learning & unstructured data

34
Deep Learning & Unstructured Data or why you should use

Upload: david-b-gonzalez

Post on 29-Jun-2015

158 views

Category:

Technology


3 download

DESCRIPTION

Morning talk given at Big Mountain Data Conference Oct 2014

TRANSCRIPT

Page 1: Deep learning & unstructured data

Deep Learning & Unstructured

Dataor why you should use ☺

Page 2: Deep learning & unstructured data

??

Page 3: Deep learning & unstructured data

The problem of doing justice to the implicit, the imponderable, and the unknown is of course not unique in politics. It is always with us in science, it is with us in the most trivial of personal affairs, and it is one of the great problems of writing and of all forms

of art. The means by which it is solved is sometimes called style. It is style

which complements affirmation with limitation and with humility; it is style which makes it possible to act effectively, but not absolutely; it is style which, in the domain of foreign policy, enables us to find harmony between the pursuit of ends essential to us, and the regard for the views, the sensibilities, the aspirations of

those to whom the problem may appear in another light; it is style which is the deference that action pays to uncertainty; it is above all style through which power defers to reason.

Page 4: Deep learning & unstructured data

Elements of styleFrom: David Gonzalez <[email protected]>Mime-Version: 1.0 (1.0)Date: Mon, 6 Mar 2011 14:57:38 -0700Delivered-To: [email protected]: <4484555894252760987@unknownmsgid>Subject: Should I take Friday off?To: Mindy Gonzalez <******@gmail.com>Content-Type: text/plain; charset=ISO-8859-1

We could go out early and maybe just get take-out for the kiddos?

??

Page 5: Deep learning & unstructured data

data?

Page 6: Deep learning & unstructured data

Gathering text

Page 7: Deep learning & unstructured data

store

{u'message_id': 4484555894252760987, u’subject’: 'Should I take Friday off', u’body’: 'We could go out early and maybe just get take-out for the kiddos. \n ??', u’time_sent’: u’Mon, 6 Mar 2011 14:57:38 -0700’, u'user_id': 1}

Page 8: Deep learning & unstructured data

Options?

Lexicon● Curated● Easy to understand● Often wrong● Not great w/

novelty

NN-style● More accurate● Domain specific● Not great w/

novelty

Page 9: Deep learning & unstructured data

Optimize Response?

Page 10: Deep learning & unstructured data
Page 11: Deep learning & unstructured data
Page 12: Deep learning & unstructured data
Page 13: Deep learning & unstructured data

Optimize Sentiment?

Page 14: Deep learning & unstructured data
Page 15: Deep learning & unstructured data
Page 16: Deep learning & unstructured data
Page 17: Deep learning & unstructured data
Page 18: Deep learning & unstructured data
Page 19: Deep learning & unstructured data
Page 20: Deep learning & unstructured data
Page 21: Deep learning & unstructured data

Optimize Emotion?

Page 22: Deep learning & unstructured data

Options?

Lexicon?

Unsupervised?

Page 23: Deep learning & unstructured data

What counts?

Subjective vs Objective statements

Polarity+, ?, -

Page 24: Deep learning & unstructured data

Sentiment “Lexicon”

Page 25: Deep learning & unstructured data

Sentiment “Lexicon”

Page 26: Deep learning & unstructured data

Emotional State

Page 27: Deep learning & unstructured data

Emotional State?

Page 28: Deep learning & unstructured data
Page 29: Deep learning & unstructured data
Page 30: Deep learning & unstructured data

Lexicon

Page 31: Deep learning & unstructured data
Page 32: Deep learning & unstructured data
Page 33: Deep learning & unstructured data
Page 34: Deep learning & unstructured data

Sponsors!