rushin shah, engineering manager, facebook at mlconf sf 2017
TRANSCRIPT
Natural Language Understanding@ Facebook Scale
ENGINEERING MANAGER, FB
Rushin Shah
Our Goal:Understand textual content with near human accuracy at Facebook scale
Friends ML Conferences
¬
HockeyThe Daily Puck
The Canadiens ran through a high-intensity practice on Wednesday ahead of Thursday's game.
1 hr •
Ron Timpany, Dan Fell and 80 others
Like Comment Share
Montreal Hockey Insider
Canadiens on verge of clinching playoff spot, Lindgren called up.
1 hr •
Bill Russell, Joe Tony and 116 others
Like Comment Share
Benoit Dumoulin was watching Vancouver Canucks vs. Montreal Canadiens.
Yes! In OT!
1 hr •
Bill Russell, Joe Tony and 234 others
Like Comment Share
VAN 3 FINAL OT 4 MTL
NLU Tasks
• Text classification
• Word classification
• Content similarity
• Entity Resolution
Text Classification
Entity: Delicious Food
Jole Simmons1 hr •
Sarah Russell and 23 others 4 Comments
Like Comment Share
Topic: Cooking
I'm trying out this new recipe for a coconut curry tonight. It looks DELICIOUS!!!
Entity: Kaepernick
Jole Simmons1 hr •
Sarah Russell and 23 others 4 Comments
Like Comment Share
Topic: Sports or Cooking ?
Last night's game was absolutely incredible. Once Curry gets cooking, there's no stopping that guy!
Word Classification
Content similarity
SIMILARITY0.75
MUCKBUCKET SUNSHINEMUSIC BAND
Parsing out Entities"DID YOU KNOW THAT MUCK BUCKET
SUNSHINE IS PERFORMING LIVE AT THE BOOM BOOM ROOM?"
TOPICLIVE EVENT
The Boom Boom Room1 hr •
Tonight before you Dance Dance Dance, join us for a special evening. Boom Boom Room Presents: Muckbucket Sunshine! Doors 8pm.
Sarah Russell and 23 others 4 Comments
Like Comment Share
• Deep Learning For NLU
• ContinuousRepresentation
• Can Solve Hard NLP Problems
Natural Language Processing (Almost) From Scratch
DeepText
DeepText Features
Multiple Tasks Multiple Languages Multiple Architectures
Model Structure Learning Algorithm
Data Loader Tokenizer Feature Extraction
Model Training DeploymentFeat Prep
DeepText Platform architecture
DeepText Tasks
Sequence LabelingClassification
DeepText Document classification
DeepText Document classification - LSTM
Hidden
MLP
Messi todayscored
HiddenHidden
EmbeddingEmbeddingEmbedding
DeepText Document classification - CNN
Messi todayscored
EmbeddingEmbeddingEmbedding
MLP
CNN
DeepText Classification: FastText
Messi todayscored
Unigram Bigram Unigram Bigram Unigram
Output
Hidden
DeepText Word classification
MLP
CNN/LSTM
Messi
Embedding
today
Embedding
Classification(Messi)
MLP
Classification(today)
scored a hat-trick
DeepText Classification: Word classification
DeepText Content Similarity
Messi todayscored
EmbeddingEmbeddingEmbedding
CNN / LSTM
Messi todayscored
EmbeddingEmbeddingEmbedding
CNN / LSTM
MLP
Ranking loss
FC Barcelona
DeepText Entity recognition and linking
Real Madrid C.F.
Document
Candidate Selection
Mention Detection Disambiguation Entity
Annotations
Entity recognition and linking Architecture
Exploring Use Cases In Facebook
For Sale
Post for SaleNot Selling
Create a Sales post to sell your items faster. Only post as a discussion if you’re notselling something.
Are You Selling Something?
Social Recs
Scale MLExperts
Reuse Models
Optimize Labels
CLUE
CLUE
Label Efficiency
Active Learning
Democratize
Single entry point for NLU
Flexibility and Scale
Deep Text
Scale MLExperts
Reuse Models
Optimize Labels
ActiveLearning
Self Service With CLUE
TrainClassifier
LabelData
CollectData
(Search)
Review
Threshold
Prec
isio
n
POOLING
...
W1 W2 W3 W4 WN
CONVOLUTIO
I want some subway or burger king
I want some subway or burger kingSlots
[{
"name": "cu:restaurant""value": "subway""start": 12"end": 19"contextldx": 0
}{
"name": "cu:restaurant""value": "burger king""start": 22"end": 32"contextldx": 0
MODELS
200+