learning by fixing: solving math word problems with weak
TRANSCRIPT
![Page 1: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/1.jpg)
Learning by Fixing: Solving Math Word Problems with Weak Supervision
![Page 2: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/2.jpg)
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
Solving Math Word Problems via Neural-Symbolic Model
+100
*/
100 2
3.5
Neural Model
2
![Page 3: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/3.jpg)
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
Solving Math Word Problems via Neural-Symbolic Model
+100
*/
100 2
3.5
Answer: 275
Neural Model
Symbolic Execution
50
175
275
3
![Page 4: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/4.jpg)
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree (Annotated):
Solving Math Word Problems with Full Supervision
+100
*/
100 2
3.5
Neural Model
Train
Train: only go through the neural module, optimize expression accuracy
Inference
4
![Page 5: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/5.jpg)
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree :
Test: go through the neural module and symbolic module, evaluate the answer accuracy
Solving Math Word Problems with Full Supervision
+100
*/
100 2
3.5
Answer: 275
Neural Model
Symbolic ExecutionTest
Inference
50
175
275
5
![Page 6: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/6.jpg)
Train: only go through the neural module, optimize expression accuracy Test: go through the neural module and symbolic module, evaluate the answer accuracy
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree :
Solving Math Word Problems with Full Supervision
+100
*/
100 2
3.5
Answer: 275
Neural Model
Symbolic Execution
Inference
Test
Discrepancy
Train
6
![Page 7: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/7.jpg)
Multiple Solutions for a given math word problem
Fully-Supervised methods: fit the given solution and cannot generate diverse solutions.7
![Page 8: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/8.jpg)
Annotating the expressions for MWPs is time-consuming.However, a large amount of MWPs with their final answers can be mined effortlessly from the internet (e.g., online forums). How to efficiently utilize these partially-labeled data without the supervision of expressions remains an open problem.
8
Fully-supervised methods: Need time-consuming annotations
![Page 9: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/9.jpg)
Solving Math Word Problems with Weak SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated)
Answer (Annotated): 275
9
![Page 10: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/10.jpg)
Solving Math Word Problems with Weak SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated):+
100
*/
100 2
2
Neural Model Inference
Answer (Annotated): 275
10
![Page 11: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/11.jpg)
Solving Math Word Problems with Weakly SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated):+
100
*/
100 2
2
Answer (Annotated):
Neural Model Inference
50
100
200
Symbolic Execution
200 ≠ 275
11
![Page 12: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/12.jpg)
Solving Math Word Problems with Weakly SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated):+
100
*/
100 2
2
Answer: 200 ≠ 275
Symbolic Execution
Neural Model Inference
50
100
200
Fixing
275
175
3.5
12
![Page 13: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/13.jpg)
Solving Math Word Problems with Weakly SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated):+
100
*/
100 2
2
Answer: 200 ≠ 275
Symbolic Execution
3.5100/
*
+2 3.5
Neural Model
157.1 ≠ 275
Inference
50
100
200
28.6 5.5
157.1
Fixing
275
175
3.5
275
50
2
13
![Page 14: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/14.jpg)
Solving Math Word Problems with Weakly SupervisionProblem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Expression Tree
(Unannotated):+
100
*/
100 2
2
Answer: 200 ≠ 275
Symbolic Execution
3.5100/
*
+2 3.5
Neural Model
157.1 ≠ 275
50
100
200
28.6 5.5
157.1
Fixing
275
175
3.5
275
50
2
Train
14
![Page 15: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/15.jpg)
Framework
15
![Page 16: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/16.jpg)
Goal-Driven Tree Structured Model[1]
16[1] A Goal-Driven Tree-Structured Neural Model for Math Word Problems. Zhipeng Xie and Shichao Sun.
![Page 17: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/17.jpg)
Goal-Driven Tree Structured Model● Word embedding + bi-directional GRU
Problem: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Word Embedding
Bi-directional GRU
17
![Page 18: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/18.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Attention
18
![Page 19: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/19.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
Attention
*
19
![Page 20: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/20.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
*
/
Attention
20
![Page 21: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/21.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
*
/
100 221
![Page 22: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/22.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
*
/
100 2+
22
![Page 23: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/23.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
*
/
100 2+
3.5 3.5 23
![Page 24: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/24.jpg)
Goal-Driven Tree Structured Model
P: A truck travels 100 kilometers in 2 hours. At this speed, if it travels for another 3.5 hours, how many kilometers will it complete for the entire journey?
*
/
100 2+
3.5 3.5 24
![Page 25: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/25.jpg)
Learning by Fixing
25[2] Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning. Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, and Song-Chun Zhu.
![Page 26: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/26.jpg)
Learning by Fixing
26
![Page 27: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/27.jpg)
Tree Regularization
1. The number of operators cannot be greater than [Size(T)/2].
2. Except the last position, the number of numeric values(quantities and constants) cannot be greater than the number of operators.
27
![Page 28: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/28.jpg)
Memory Buffer
28
![Page 29: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/29.jpg)
Experiment● Dataset:
Math23K, 23161 math word problems
● Evaluation Metric:
Answer accuracies of all the top-1/3/5 predictions using beam search
● Inference Models:
Seq2Seq, Goal-Driven Tree-Structured Model (GTS)
● Learning Strategies:
REINFORCE, MAPO[3], LBF (Learning by Fixing), LBF-w/o-M (Fixing without Memory)
[3] Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing. Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le and Ni Lao. 29
![Page 30: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/30.jpg)
Top-1 Answer Accuracy
30
![Page 31: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/31.jpg)
Diverse Solutions with Memory Buffer, Ablative Studies
31
![Page 32: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/32.jpg)
Qualitative Study
32
![Page 33: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/33.jpg)
Conclusions & Future Works● We propose a weakly-supervised paradigm for learning MWPs and a novel
learning-by-fixing framework to boost the learning.● For future work, we will prevent generating equivalent or spurious solutions during
training, possibly by making the generated solution trees more interpretable with semantic constraints. (See also our newest work[4]!)
● A weakly-supervised large-scale dataset on math word problems would be beneficial for this line of research.
[4] "SMART: A Situation Model for Algebra Story Problems via Attributed Grammar". Yining Hong, Qing Li, Ran Gong, Daniel Ciao, Siyuan Huang, Song-Chun Zhu.
33
![Page 34: Learning by Fixing: Solving Math Word Problems with Weak](https://reader030.vdocuments.us/reader030/viewer/2022012020/61688dd5d394e9041f708894/html5/thumbnails/34.jpg)
You are welcomed to visit our project pages!
The project page of this paper: https://evelinehong.github.io/lbf-site/
For more details about the fixing mechanism: https://liqing-ustc.github.io/NGS/
For interpretable math word problems solving: https://evelinehong.github.io/smart-site/
34