from natural language specifications to program...
TRANSCRIPT
![Page 1: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/1.jpg)
From Natural Language Specifications to Program Input Parsers
Tao Lei,
Fan Long, Regina Barzilay, Martin Rinard
CSAIL, MIT
1
![Page 2: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/2.jpg)
Translating Natural Language to Input Parser
2
Input Specification:
Input Parser:
Defines the format of input data
- The input starts with a line containing two integers n and r. - This is followed by n lines, each containing two integers xi, yi, giving the coordinates of the polygon vertices.
Part of a program that reads and stores data
Two Input Examples:
3 6 0 4 0 0 5 1
4 10 -8 2 8 14 0 14 0 6
int n, r, x[], y[];
Scanner scanner = new Scanner(new File(“input.txt”));
n = scanner.nextInt(); r = scanner.nextInt();
x = new int[n]; y = new int[n]; for (int i = 0; i < n; i++) { x[i] = scanner.nextInt(); y[i] = scanner.nextInt(); }
![Page 3: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/3.jpg)
Translating Natural Language to Input Parser
3
Input Specification:
Input Parser:
Defines the format of input data
- The input starts with a line containing two integers n and r. - This is followed by n lines, each containing two integers xi, yi, giving the coordinates of the polygon vertices.
Part of a program that reads and stores data
Two Input Examples:
3 6 0 4 0 0 5 1
4 10 -8 2 8 14 0 14 0 6
int n, r, x[], y[];
Scanner scanner = new Scanner(new File(“input.txt”));
n = scanner.nextInt(); r = scanner.nextInt();
x = new int[n]; y = new int[n]; for (int i = 0; i < n; i++) { x[i] = scanner.nextInt(); y[i] = scanner.nextInt(); }
Goal: generating input parser by reading natural language
![Page 4: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/4.jpg)
Motivation
4
• Reading and processing data is a common task
• Writing input parsers is mechanical, tedious and time-consuming
John ate an apple NN VB DT NN SUBJ ROOT MOD OBJ 2 0 4 2 The dog barks DT NN VB MOD SUBJ ROOT 2 3 0
MST dependency data format
This DT is VBZ a DT short JJ sentence NN . . So RB is VBZ this DT
1 Cathy Cathy N N … 2 su 2 zag zie V V … 0 ROOT 3 hen hen Pron Pron … 2 obj1 4 wild wild Adj Adj … 5 mod 5 zwaaien zwaai N N … 2 vc 6 . . Punc Punc … 5 punct
…
POS tagger data format
CONLL dependency data format
![Page 5: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/5.jpg)
Motivation
5
• Reading and processing data is a common task
• Writing input parsers is mechanical, tedious and time-consuming
Parser Generator (our model)
Input Parser (in C++, Java, …)
Input Example:
10 abc xyz uvw efg …
Input Specification:
“The input is one integer followed by a list of strings.”
Allows natural language as the interface to specify input
![Page 6: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/6.jpg)
Motivation
6
• Reading and processing data is a common task
• Writing input parsers is mechanical, tedious and time-consuming
Parser Generator (our model)
Input Parser (in C++, Java, …)
Input Example:
10 abc xyz uvw efg …
Input Specification:
“The input is one integer followed by a list of strings.”
Allows natural language as the interface to specify input
Advantage: reducing programming effort and the chance of making code mistakes
![Page 7: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/7.jpg)
How to Translate NL to Input Parser?
7
• Need an abstraction that connects NL and input parser
Input Specification: The input consists of multiple sentences. • The first line of each sentence is the list of
words in the sentence; • The second line of each sentence contains
the POS tokens; • The third line are dependency labels; • The last line are integers representing the
positions of each word’s parent.
Input Parser: sentence = [ ]; with open(“input.txt”) as fin: line = fin.readline().strip(); while line: if line != “”: word = line.split(); pos = fin.readline().split(); label = fin.readline().split(); parent = fin.readline().split(); parent = [ int(x) for x in parent ]; sentence.append( (word, pos, label, parent) ); line = fin.readline().strip();
?
![Page 8: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/8.jpg)
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
8
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 9: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/9.jpg)
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
Input
9
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 10: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/10.jpg)
Sentences
Input
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
10
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 11: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/11.jpg)
Sentences
Input
Words
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
11
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 12: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/12.jpg)
Sentences
Input
Words POS
Tokens
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
12
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 13: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/13.jpg)
Sentences
Input
Words POS
Tokens Labels
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
13
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 14: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/14.jpg)
Sentences
Input
Words POS
Tokens Labels
Position Integers
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
14
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
![Page 15: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/15.jpg)
Sentences
Input
Words POS
Tokens Labels
Position Integers
Specification Tree
Input Example: John ate an apple
NN VB DT NN
SUBJ ROOT MOD OBJ
2 0 4 2
The dog barks
DT NN VB
MOD SUBJ ROOT
2 3 0
…
15
How to Translate NL to Input Parser?
• Need an abstraction that connects NL and input parser
• Specification tree of nested input formats
![Page 16: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/16.jpg)
16
How to Translate NL to Input Parser?
Specification Tree
Input Specification
Input Parser
The input parser is deterministically generated from the specification tree.
• Need an abstraction that connects NL and input parser
• Specification tree of nested input formats
![Page 17: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/17.jpg)
17
How to Translate NL to Input Parser?
The input parser is deterministically generated from the specification tree.
Focus: translating input specifications into specification trees
• Need an abstraction that connects NL and input parser
• Specification tree of nested input formats
Specification Tree
Input Specification
Input Parser
![Page 18: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/18.jpg)
How to Translate NL to Specification Tree?
18
Specification Tree
Input Specification Specification tree is a dependency tree
over noun phrases in the NL specification.
Input Specification: The input consists of multiple sentences.
• The first line of each parse is the list of words in the sentence;
• The second line of each parse contains the POS tokens;
• The third line are dependency labels; • The last line are integers representing
the positions of each word’s parent.
Sentences
Input
Words POS
Tokens Labels
Position Integers
Task: translation as an NLP problem
![Page 19: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/19.jpg)
Learning Scenario
19
N input specifications 𝒘 = 𝑤1,… , 𝑤𝑁
Input
The input consists of a single test case. A test case consists of two lines. The first line contains an integer n indicating the number of molecule types. The second line contains n eight-character strings, each describing a single type of molecule, separated by single spaces. Each string consists of four two-character connector labels
some input examples for each specification
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
specification trees 𝒕 = 𝑡1,… , 𝑡𝑁
No human annotation
corresponding input parsers
𝒕 ~ 𝑃 𝒕 𝒘
![Page 20: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/20.jpg)
Learning Scenario
20
N input specifications 𝒘 = 𝑤1,… , 𝑤𝑁
Input
The input consists of a single test case. A test case consists of two lines. The first line contains an integer n indicating the number of molecule types. The second line contains n eight-character strings, each describing a single type of molecule, separated by single spaces. Each string consists of four two-character connector labels
some input examples for each specification
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
Input Example:
3 A+00A+A+ 00B+D+A- B-C+00C+
specification trees 𝒕 = 𝑡1,… , 𝑡𝑁
No human annotation
corresponding input parsers
𝒕 ~ 𝑃 𝒕 𝒘 Idea: learning from feedback -- testing input parser on input examples
![Page 21: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/21.jpg)
Key Intuitions
21
a correct tree should read all input examples successfully
5 -8 8 0 0 -8
a list of integers?
a list of strings?
a list of integer pairs?
…
Input Example Possible Interpretations
• Necessary but NOT sufficient condition • False-positive parsers
Many input parsers can read the same input
![Page 22: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/22.jpg)
Key Intuitions
22
the correct trees should share common features
The input contains an integer
Test case contains several strings
Each line starts with two numbers
X contains Y
X starts with Y
a correct tree should read all input examples successfully
the input
an integer
test case
several strings
Patterns Example Sentences Tree Structures
![Page 23: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/23.jpg)
Bayesian Generative Model
𝑃 𝜃 ⋅ 𝑃 𝑡𝑖 ⋅ 𝑃 𝑤𝑖 𝑡𝑖; 𝜃𝑖
(i) Generating Parameters
𝜃⋅~ 𝐷𝑖𝑟𝑖𝑐ℎ𝑙𝑒𝑡 𝜶
(ii) Generating Specification Trees
𝑃 𝑡𝑖 ∝
1
𝜖
parser of t i read input examples successfully
otherwise
(iii) Generating Feature Observations
𝑃 𝑤 𝑖 𝑡 𝑖;𝜃 = 𝜃𝑓𝑓∈𝜙 𝑤𝑖 ,𝑡𝑖
𝜙 𝑤 𝑖, 𝑡 𝑖 : set of features over (w i, t i )
Idea: encode both intuitions in our model 23
the correct trees should share common features
a correct tree should read all input examples successfully
![Page 24: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/24.jpg)
Inference: Gibbs Sampling
24
𝑡1 𝑡2 ⋯ 𝑡𝑖 ⋯ 𝑡𝑁
update specification tree t i for the i-th input specification
Sample from conditional probability:
𝑡𝑖 ~ 𝑃 𝑡𝑖|𝒘, 𝒕−𝑖 Intractable
𝒕 ~ 𝑃 𝒕 𝒘 = 𝑃 𝒕,𝜃 𝒘𝜃
![Page 25: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/25.jpg)
Inference: Gibbs Sampling
25
𝑡1 𝑡2 ⋯ 𝑡𝑖 ⋯ 𝑡𝑁
(i) Estimate current parameters
(ii) Sample a new tree
(iii) Apply Metropolis-Hastings rule 𝜃∗ = 𝐸 𝜃|𝒘, 𝒕−𝑖
𝑡′~ 𝑄 𝑡′ ∝ 𝑃 𝑤 𝑖|𝑡′; 𝜃∗
𝑡𝑖 ≔ 𝑡′ with probability:
min 1,𝑃(𝑡𝑖)𝑄(𝑡′)
𝑃 𝑡′ 𝑄 𝑡𝑖
𝒕 ~ 𝑃 𝒕 𝒘 = 𝑃 𝒕,𝜃 𝒘𝜃
update specification tree t i for the i-th input specification
![Page 26: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/26.jpg)
Experiments
26
Sentences: 424
Vocabulary: 781
# of Sent. in Document 1 ~ 8
Avg. Sent. Length 17.3
Text Statistics:
Domain:
Programming contest (ACM-ICPC)
Training Data: 106 input specifications 100 input examples for each
relative clauses in sentences
![Page 27: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/27.jpg)
Evaluation Metrics
27
Recall:
Precision:
F-Score:
# correct specification trees
# positive specification trees
# correct specification trees
# input specifications
2 × Precision × Recall
Precision + Recall
![Page 28: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/28.jpg)
Baseline Models
28
Aggressive (Clarke et al. 2010)
Trains a discriminative structure learner (SVMStruct) using all “positive” specification trees obtained in previous iteration; uses the learner to find the most plausible trees in the next iteration
No Learning
Does not learn feature parameters; randomly samples the specification tree until successfully reads all input examples
Full Model - Oracle
An “oracle” feedback tells our full model whether the specification tree is correct or not
Aggressive - Oracle
Trains SVM using perfect oracle supervision signal
![Page 29: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/29.jpg)
Overall Performance
80.00%
66.70%
54.50%
0% 20% 40% 60% 80% 100%
Full Model
Aggressive
No Learning
29
F-Score
• Search space is exponential, and is large on difficult specifications • Cannot distinguish between correct parsers and false-positive parsers
![Page 30: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/30.jpg)
Overall Performance
80.00%
66.70%
54.50%
0% 20% 40% 60% 80% 100%
Full Model
Aggressive
No Learning
30
F-Score
• Using false-positive parsers to train SVM will hurt the performance
![Page 31: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/31.jpg)
Overall Performance
80.00%
66.70%
54.50%
0% 20% 40% 60% 80% 100%
Full Model
Aggressive
No Learning
31
F-Score
• Learns from feedback and feature observations in a joint, complementary fashion
![Page 32: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/32.jpg)
Comparison with Oracles
89.00%
84.10%
80.00%
50% 60% 70% 80% 90% 100%
Aggressive-Oracle
Full Model-Oracle
Full Model
32
F-Score
![Page 33: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/33.jpg)
Comparison with Oracles
89.00%
84.10%
80.00%
50% 60% 70% 80% 90% 100%
Aggressive-Oracle
Full Model-Oracle
Full Model
33
• Discriminative model is better at learning from strong supervision • Generative model is itself much more constrained
F-Score
![Page 34: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/34.jpg)
Learning Curve as a Function of # Input Examples
34
• May not be possible to obtain so many input examples • Retains high performance when just one example is available
totally unsupervised generative model
![Page 35: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/35.jpg)
Conclusion
• A new problem in addition to generating database queries or regular expressions from natural language
• Our method can learn to ground natural language descriptions of input data formats
Code and data available at:
http://groups.csail.mit.edu/rbg/code/nl2p
35
![Page 36: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/36.jpg)
36
![Page 37: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/37.jpg)
37
![Page 38: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/38.jpg)
38
![Page 39: From Natural Language Specifications to Program …people.csail.mit.edu/taolei/papers/acl2013-slides.pdfFrom Natural Language Specifications to Program Input Parsers Tao Lei, Fan Long,](https://reader033.vdocuments.us/reader033/viewer/2022042320/5f09b63c7e708231d428273b/html5/thumbnails/39.jpg)
39