fmcad paper sygus-comp organizers: m. raghothaman, s. … · syntax-guided program synthesis common...
TRANSCRIPT
![Page 1: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/1.jpg)
R. Alur, D. Fisman, R. Singh, A. Solar-Lezama
R. Alur, R. Bodik, G. Juniwal, M. Martin, M. Raghothaman, S. Seshia, R. Singh, A. Solar-Lezama, E. Torlak, A. Udupa
FMCAD paper SyGuS-Comp Organizers:
![Page 2: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/2.jpg)
Traditional Program Synthesis
Given a specification S, produces a program P satisfying S.
Program Synthesis
Synthesizer
Specification SHigh Level“WHAT”
Program PLow Level“HOW”
![Page 3: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/3.jpg)
New Trends in Synthesis
Motivation: Tractability
Combine
human expert insights with
computers exhaustiveness & rapidness
Benefit progress SAT & SMT Solvers
Emerging Synthesis Tools
Synthesizer
Program PLow Level“HOW”
Augment the specification S, with some syntactic restrictions R on the domain of possible solutions.
Specification Sgiven by
logical constraints
Syntactic restrictions R on the
solutions domain
![Page 4: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/4.jpg)
SAT and SMT Solvers
Is formula φ, satisfiable?
Satisfiability
SAT/SMT Solver
Logical formula φ(a set of constraints)
Unsastisfiable / Satisfiable+ satisfying assignment
SATFormulas use Boolean variables + Logical connectives
SMT(Satisfiability ModuloTheories) Formulas are interpreted in some theory, e.g. BitVectors, Integers, Unintepreted functions.
![Page 5: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/5.jpg)
From Satisfiability to Synthesis
Recent trends in program synthesis:
SAT/SMT Solver
SyntacticRestrictions
on solution domain
Problem(verif/synth nature)
Problem(verif/synth nature)
![Page 6: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/6.jpg)
Syntax-Guided Program Synthesis
Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al)
FlashFill (Gulwani et al)
Super-optimization (Schkufza et al)
Invariant generation (Many recent efforts…)
TRANSIT for protocol synthesis (Udupa et al)
Oracle-guided program synthesis (Jha et al)
Implicit programming: Scala^Z3 (Kuncak et al)
Auto-grader (Singh et al)
Synthesizer
j E
P
Program optimization
Program sketching
Programming by examples
Invariantgeneration
But no way to have a generic solver for all
![Page 7: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/7.jpg)
Talk Outline
Motivation for Syntax-Guided Synthesis
Formalization of Syntax-Guided Synthesis
Solution Strategies
Conclusions + SyGuS Competition
![Page 8: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/8.jpg)
Syntax-Guided Synthesis (SyGuS) Problem
Fix a background theory T: fixes types and operations
Function to be synthesized: name f along with its type General case: multiple functions to be synthesized
Inputs to SyGuS problem: Specification j
Typed formula using symbols in T + symbol f
Context-free grammar G
Characterizing the set of allowed expressions E (in theory T)
Computational problem: Find expression e in E such that j[f/e] is valid (in theory T)
Synthesizer
j E
P
Theory T
f
ft1
+0
Grammar
f1
f2
![Page 9: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/9.jpg)
SyGuS - example Theory QF-LIA
Types: Integers and BooleansLogical connectives, Conditionals, and Linear arithmeticQuantifier-free formulas
Function to be synthesized f (int x, int y) : int
Specification: (x ≤ f(x,y)) & (y ≤ f(x,y)) & ( f(x,y)=x | f(x,y)=y )
Candidate Implementations: Conditional expressions with comparisons
Term := x | y | Const | If-Then-Else (Cond, Term, Term)Cond := Term <= Term | Cond & Cond | ~Cond | (Cond)
Possible solution:If-Then-Else (x ≤ y, y, x)
Candidate Implementations: Linear expressions
LinExp := x | y | Const | LinExp + LinExp | LinExp - LinExp
No solution exists
![Page 10: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/10.jpg)
SyGuS - example Theory QF-LIA
Types: Integers and BooleansLogical connectives, Conditionals, and Linear arithmeticQuantifier-free formulas
Function to be synthesized f (int x, int y) : int
Specification: (x ≤ f(x,y)) & (y ≤ f(x,y)) & ( f(x,y)=x | f(x,y)=y )
Candidate Implementations: Conditional expressions with comparisons
Term := x | y | Const | If-Then-Else (Cond, Term, Term)Cond := Term <= Term | Cond & Cond | ~Cond | (Cond)
Possible solution:If-Then-Else (x ≤ y, y, x)
Common theme to many recent efforts
Sketch (Bodik, Solar-Lezama et al)
FlashFill (Gulwani et al)
Super-optimization (Schkufza et al)
Invariant generation (Many recent efforts…)
TRANSIT for protocol synthesis (Udupa et al)
Oracle-guided program synthesis (Jha et al)
Implicit programming: Scala^Z3 (Kuncak et al)
Auto-grader (Singh et al)
![Page 11: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/11.jpg)
Talk Outline
Motivation for Syntax-Guided Synthesis
Formalization of Syntax-Guided Synthesis
Solution Strategies
Conclusions + SyGuS Competition
![Page 12: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/12.jpg)
SyGuS as Active Learning
12
LearningAlgorithm
VerificationOracle
Initial examples I
Fail Success
Candidate
Expression
Counterexample
Concept class: Set E of expressions
Examples: Concrete input values
j[f/e]
![Page 13: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/13.jpg)
SyGuS Solutions
CEGIS - Counter-Example Guided Inductive Synthesis approach (Solar-Lezama, Seshia et al)
Related work: Similar strategies for solving quantified formulas and
invariant generation
Learning strategies based on:
Enumerative (search with pruning): Udupa et al (PLDI’13)
Symbolic (solving constraints): Gulwani et al (PLDI’11)
Stochastic (probabilistic walk): Schkufza et al (ASPLOS’13)
![Page 14: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/14.jpg)
SyGuS - Recap
Program optimization
Program sketching
Programming by examples
Invariantgeneration
![Page 15: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/15.jpg)
SyGuS - Recap
Program optimization
Program sketching
Programming by examples
Invariantgeneration
SyGuS IF
Generic Solvers
Benchmark + Compare + Compete
=> Boost improvement
???
![Page 16: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/16.jpg)
SyGuS Competition
Initiated a competition of SyGuS solvers at FLoC Olympic Games, 30 June 2014
Competition is going to be offline, using StarExec.
Single track, solvers should support full SyGuS-IF format for the theories of bit-vector and integer linear arithmetic.
Caveat for “let” – Solvers do not supporting “let” will be evaluated against other such solvers
Benchmarks: Bit-manipulation programs from Hacker’s delight
Integer arithmetic: Find max, search in sorted array
Challenge problems such as computing Morton’s number
![Page 17: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/17.jpg)
SyGuS-Comp
Awards: Kurt Gödel Silver Medals
$1000 sponsored graciously by Microsoft Research
Important Dates: Solver and Benchmark registration is now open
Solver registration deadline: 15 June 2014
Competition takes place: 30 June 2014
SyGuS session on SYNT: 23-24 July 2014
FLoC Award ceremony: 21 July 2014
![Page 18: FMCAD paper SyGuS-Comp Organizers: M. Raghothaman, S. … · Syntax-Guided Program Synthesis Common theme to many recent efforts Sketch (Bodik, Solar-Lezama et al) FlashFill (Gulwani](https://reader033.vdocuments.us/reader033/viewer/2022041902/5e61660a68266a40900556d7/html5/thumbnails/18.jpg)
The End
Thank you!Questions?