[ 1 ] claus brabrand, itu banana algebra diku, ku, denmark may 11, 2010 banana algebra: jacob...

[ 1 ]Claus Brabrand, ITU BANANA ALGEBRA DIKU, KU, Denmark May 11, 2010

Banana Algebra:Banana Algebra:

Jacob Andersen[ [email protected] ]

Aarhus University

Claus Brabrand[ [email protected] ]

IT University of Copenhagen

Syntactic Language Extension via an Algebra of Languages and Transformations

[ 3 ]Claus Brabrand, ITU BANANA ALGEBRA May 11, 2010DIKU, KU, Denmark

The Banana Algebra CaféThe Banana Algebra Café

The "Banana Algebra Café":Located in Costa Rica

City of Cahuita (pop. 3,000):

Cahuita!Cahuita!

BananaBreakfast!BananaBreakfast!


Introduction: "What is a Banana?"

Bananas for Language Transformation

Language Extension Pattern

Banana Algebra

Examples

Implementation

Related Work

Conclusion

OutlineOutline


What is a 'Banana' ?What is a 'Banana' ?

Datatype; "list":

Banana ("sum-of-list"):

Implicit recursion on input structure

Separation of recursion and evaluation

bottom-up re-combination of intermediate results

list = Num int | Cons int * list

[Num n] = n[Cons n l] = n + [l]

list int

(aka. "Catamorphism" )

(| n.n , (n,l ).n+l |) Another Ex.:"length-of-list"


Language TransformationLanguage Transformation

Bananas (statically typed):

Source language: 'LS'

Target language: 'LT'

Nonterminal-typing: ' '

Reconstructors: 'c '

list = Num int | Cons int * list

tree = Nil | Leaf int | Node tree * tree

[Num n] = Leaf n[Cons n l] = Node (Leaf n) [l]

[list -> tree]

(| LS -> LT [] c |)

Type-check'able!

LS -> LT


Banana properties:Simple (corresponds to: “simple recursion”)Safe (syntactically safe + always terminate)Efficient (linear time in size of input + output)(Expressive) (…enough for interesting extensions)

Banana Algebra “for free” (16 banana ops):ModularIncrementalSimpleSafeEfficient(Expressive)

Statically reduce: Banana Algebra (term) Banana (const)

Banana PropertiesBanana Properties"The metafront System: Safe and Extensible Parsing and Transformation" [ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003, SCP J. 2007 )

"Growing Languages with Metamorphic Syntax Macros" [ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 )

Been

aro

un

d f

or

man

y y

ears

We n

ow

pro

pose





Banana Algebra

Examples

Implementation

Related Work

Conclusion

OutlineOutline


Language Extension PatternLanguage Extension Pattern

[var V] = var V

[lam V E] = lam V [E ]

[app E1 E2] = app [E1] [E2]

[zero] = lam z (var z)

[succ E] = lam s [E ]

[pred E] = app [E ] (lam z (var z))

Numeral extension: Lambda-Calculus:

Nonterminal typing:

Reconstructors:

[Exp -> Exp]

Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp

Exp : var Id : lam Id * Exp : app Exp * Exp

'LT''LS'

' '

'c '

(| LS -> LT [] c |)Catamorphism:

Using very simple numeral encoding


Algebraic SolutionAlgebraic Solution


Exp : zero : succ Exp : pred Exp

(| ln -> l [Exp -> Exp]

[zero] = lam z (var z) [succ E] = lam s [E ] [pred E] = app [E ] ...|)

idx

+

lnl

ln l

ln+l l

l l


Extending JavaExtending Java

Java grammar:( 575 lines !!! )...

Stm : "repeat" Stm "until" "(" Exp ")" ";"

(| repeat -> java [Stm -> Stm, Exp -> Exp]

[repeat S E] = do-while [S] (not [E])|)

idx

+

repeatjava

repeat java

java+repeat java

java java


Banana AlgebraBanana Algebra

Languages (L):

lv

L \ L

L + L

src ( X )

tgt ( X )

let v = L in L

letx w = X in L

Transformations (X):

xw

X \ L

X + X

X X

idx ( L )

let v = L in X

letx w = X in X

(| L -> L [] c |){ CFG }


Algebraic LawsAlgebraic Laws

Idempotency of '+':

Commutativity of '+':

Associativity of '+':

Source-identity:

…

L L + L

L1 + L2 L2 + L1

L1 + (L2 + L3) (L1 + L2) + L3

L src(idx(L)) L tgt(idx(L))Target-identity:





Banana Algebra

Examples

Implementation

Related Work

Conclusion

OutlineOutline


Example RevisitedExample Revisited

{ Id = [a-z] [a-z0-9]* ;

Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ;}

{ Exp.zero : "zero" ; Exp.succ : "succ" "(" Exp ")" ; Exp.pred : "pred" "(" Exp ")" ;}

--- "l.l" --- --- "ln.l" ---

let l = "l.l"in let ln = "ln.l"in idx(l) + (| ln -> l [Exp -> Exp] Exp.zero = '\z.z' ; Exp.succ = '\s.$1' ; Exp.pred = '($1 \z.z)' ; |)

--- "ln2l.x" ---


Numerals + BooleansNumerals + Booleans

l

lb l idx

+lb+l l

l l

lb l

ln l idx

+ln+l l

ln

+l+ln+lb l

…with Nums …with Bools

…with Nums & Bools?


Java + RepeatJava + Repeat

let java = "java.l"in let repeat = "repeat.l"in idx(java) + (| repeat -> java [Exp -> Exp, Stm -> Stm]

Stm.repeat = 'do $1 while (!($2));' ; |)

{ Stm.repeat : "repeat" Stm "until" "(" Exp ")" ";" ; }

7 lines !

{ Java ... "try" Stm "catch" ... Name.id : Id ; }

575 lines--- "java.l" ---

--- "repeat.l" ---

--- "repeat2java.x" ---


Concrete vs. Abstract SyntaxConcrete vs. Abstract Syntax

Exp.or : Exp1 "||" Exp ; .exp1 : Exp1 ;Exp1.and : Exp2 "&&" Exp1 ; .exp2 : Exp2 ;Exp2.add : Exp3 "+" Exp2 ; .exp3 : Exp3 ; Exp7.neg : "!" Exp8 ; .exp8 : Exp8 ;Exp8.par : "(" Exp ")" ; .var : Id ; .num : IntConst ;

Exp (with explicit assoc./prec.):Stm.repeat = Stm.do(<1>, Exp.exp1( Exp1.exp2( Exp2.exp3( Exp3.exp4( Exp4.exp5( Exp5.exp6( Exp6.exp7( Exp7.neg( Exp8.par(<2>) ))))))))) ;

Abstract syntax:

Stm.repeat = 'do $1 while (!($2));' ;

Concrete syntax:

NB: Tool supports BOTH ! (unambiguous: concrete abstract)


"FUN" Example"FUN" Example

Lambda Calculus

Nums→

Unsigned arithmetic + booleans + definitions + pairs

Bools→ Defs→ Pairs→+ + +

Fun

Literals

Literals→Nums

Fun grammar transform

The "FUN" Language: used for Teaching Functional Programming

Basically The Lambda Calculus with…: numerals, booleans, arithmetic, boolean logic, local definitions, pairs, literals, lists, signs, comparisons, dynamic types, fixed-point combinators, …

(at Aarhus University)


Literals

Literals→Nums


Lambda Calculus

Nums→



Fun

Fun grammar transform

Signed arith→Nums

Literals→Nums

Fun grammar transform FunSigned GT+

Fun FunSigned+Component re-use



Lambda Calculus

Nums→



Fun GT FunSigned GT+

Fun FunSigned+ FunCompare FunTypesafe+ +

FunCompare GT+ FunTypesafe GT+

245x Banana Algebra ops 4 MB Banana !


"FUN" Usage Statistics"FUN" Usage Statistics

Usage statistics (245x operators) in "FUN":58x { …cfg… } Constant languages51x "file.l" Language inclusions28x L + L Language additions23x v Language variables17x (|L L [] c|) Constant transformations17x X + X Transformation additions14x "file.x" Transformation inclusions10x let-in Local definitions 9x idx(L) Identity transformations 8x X X Compositions 4x L \ L Language restriction 4x w Transformation variables 2x src(X) Source extractions


Incremental DevelopmentIncremental Development

let l = "l.l" in idx(l) + (| "ln.l" -> l [Exp -> Exp] Exp.zero : '\z.z' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 \z.z)' ; |)

let l = "l.l" in idx(l) + (| "li.l" -> l [Exp -> Exp] Exp.id : '\z.z' ; |)

{ Exp.zero : "zero" ; Exp.succ : "succ" Exp ; Exp.pred : "pred" Exp ; }

{ Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; }

--- "ln.l" ---

--- "ln2li.x" ---

--- "l.l" ---

--- "ln2l.x" ---let l = "l.l" in idx(l) + (| ln -> l+"li.l" [Exp -> Exp] Exp.zero : 'id' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 id)' ; |)

{ Exp.id : "id" ; }--- "li.l" ---

--- "li2l.x" ---

"li2l.x" o "ln2li.x"

--- "ln2l.x" ---

EXERCISE


Example cont'dExample cont'd

Both statically reduce to same catamorphism:(|

Exp.app : Exp.app($1, $2) ; Exp.lam : Exp.lam($1, $2) ; Exp.pred : Exp.app($1, Exp.lam(Id("z"), Exp.var(Id("z")))) ; Exp.succ : Exp.lam(Id("x"), $1) ; Exp.var : Exp.var($1) ; Exp.zero : Exp.lam(Id("z"), Exp.var(Id("z"))) ;|)

{ Id = [a-z] [0-9a-z]* ;

Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.var : Id ; }

{ Id = [a-z] [0-9a-z]* ;

Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.pred : "pred" Exp ; Exp.succ : "succ" Exp ; Exp.var : Id ; Exp.zero : "zero" ; }

->

[Exp -> Exp, Id->Id]


Other ExamplesOther Examples

Self-Application (The tool on itself!):

SQL embedding (in <bigwig>):

My-Java (endless variations):

[L1 << L2] = '(L1 \ L2) + L2' [X1 << X2] = '(X1 \ src(X2)) + X2'

Stm.select = 'factor (<2>) { if (<3>) return ( # \+ (<1>) ); }'

java( + sql) ( \ loops) o syntaxe_francais


ImplementationImplementation

[ http://www.itu.dk/people/brabrand/banana-algebra/ ]

The 'Banana Algebra' Tool:(3,600 lines of O'Caml)

Uses (underlying technologies): 'dk.brics.grammar': for parsing, unparsing, and ambiguity analysis ! 'XSugar': for transformation: "concrete syntax abstract XML syntax" 'XSLT': for transformation: "XML XML"





Banana Algebra

Examples

Implementation

Related Work

Conclusion

OutlineOutline


Related Work (I/III)Related Work (I/III)

Macro Systems:

"The metafront System: Safe and Extensible Parsing and Transformation" [ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003 , SCP J. 2007 )

"Growing Languages with Metamorphic Syntax Macros" [ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 )


Related Work (II/III)Related Work (II/III)

Attribute Grammars:Language transformation (and extension)…

…via computation on AST's

(using "inherited" or "synthesized" or … attributes)

E.g., Eli, JastAdd, Silver, …

Rewrite Systems:Language transformation (and extension)…

…via syntactic rewriting, using encodings…:

gradually rewrite "S-syntax" to "T-syntax"

gradually rewrite "S-syntax" to "T-syntax"

E.g., Elan, TXL, ASF+SDF, Stratego/XT, …

S T

S T

Both; compared to bananas: More ambitious (expressivity) No termination guarantees (safety) Transformation "indirect" (simplicity)


Related Work (III/III)Related Work (III/III)

Functional Programming:Catas mimicked by "disciplined style" of fun. programming

…aided by:Traversal functions (auto-synthesized from datatypes)

Combinator libraries

"Shortcut fusion" (to eliminate ' ' at compile-time)

Category Theory:A lot of this work can be viewed as Category Theory:

Basically yeolde issue:GPL vs. DSL


ConclusionConclusion

IF bananas are sufficiently:(Expressive)

THEN you get…:

Banana Algebra “for free” (16 banana ops):IncrementalModularSimpleSafeEfficient

"Niche"

Statically reduce: Banana Algebra (term) Banana (term)


BONUS SLIDESBONUS SLIDES

- Reduction Semantics -

"Syntactic Language Extension via an Algebra of Languages and Transformations" [ Jacob Andersen | Claus Brabrand ] ( ITU Technical Report, Dec. 2008 )

If you want all the details:


Reduction SemanticsReduction Semantics

Environments:

Reduction relations:

Abbreviations: ...as a short-hand for:

...as a short-hand for:

ENVL = VARL EXPL

ENVL ENVX EXPL EXPL 'L'

ENVX = VARX EXPX

ENVL ENVX EXPX EXPX 'X'

, |- L L l (,,L,l) 'L'

(,,X,x) 'X', |- X X x

environment of languages

environment of transformations


Semantics (L)Semantics (L)

, l L l[CONL] , v L (v)

[VARL]

, L \ L' L l l'[RESL]

, L' L l', L L l

, L + L' L l l'[ADDL]

, L' L l', L L l

l

ll ~ l'l

lwfl


Semantics (L)Semantics (L)

, src (X) L lS

[SRCL], X X (| lS -> lT [] c |)

, tgt (X) L lT

[TGTL], X X (| lS -> lT [] c |)

, let v=L in L' L l'[LETL]

[v=l], L' L l', L L l



- More Examples -


Numeral & Boolean ExtensionNumeral & Boolean Extension

Numeral Extension (catamorphism):

Boolean Extension (catamorphism):

[var V] = var [V ]

[lam V E] = lam [V ] [E ]

[app E1 E2] = app [E1] [E2]

[zero] = lam z (var z)

[succ E] = lam s [E ]

[pred E] = app [E ] (lam z (var z))

[var V] = var [V ] [lam V E] = lam [V ] [E ] [app E1 E2] = app [E1] [E2] [true] = lam a (lam b (var a)) [false] = lam a (lam b (var b))[if E1 E2 E3] = app (app [E1] [E2]) [E3]

Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp


Exp : var Id : lam Id * Exp : app Exp * Exp : true : false : if Exp Exp Exp



Lambda with BooleansLambda with Booleans

Exp : true : false : if Exp Exp Exp

(| lb -> l [Exp -> Exp]

[true] = '\a.\b . a' [false] = '\a.\b . b'[if E1 E2 E3] = '(([E1] [E2]) [E3])'

|)

idx

+

lbl

lb l

lb+l l

l l



Incremental DevelopmentIncremental Development

let l = "l.l" in idx(l) + (| "ln.l" -> l [Exp -> Exp] Exp.zero : '\z.z' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 \z.z)' ; |)

let l = "l.l" in idx(l) + (| "li.l" -> l [Exp -> Exp] Exp.id : '\z.z' ; |)

{ Exp.zero : "zero" ; Exp.succ : "succ" Exp ; Exp.pred : "pred" Exp ; }

{ Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; }

--- "ln.l" ---

--- "ln2li.x" ---

--- "l.l" ---

--- "ln2l.x" ---let l = "l.l" in idx(l) + (| ln -> l+"li.l" [Exp -> Exp] Exp.zero : 'id' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 id)' ; |)

{ Exp.id : "id" ; }--- "li.l" ---

--- "li2l.x" ---

"li2l.x" o "ln2li.x"

--- "ln2l.x" ---


Usage ScenariosUsage Scenarios

Programmers:May extend existing languages (~ syntax macros)

Developers:May embed DSLs into host languages (SQL in Java)

Developers (and teachers):May incrementally specify multi-layered languages

Compiler writers:May rely on tool and implement only a small core

(and then specify the rest externally as extensions)



- Parsing & Error Reporting -


ParsingParsing

Parsing (XSugar):Variant of Earley's algorithm: O ( ||3 )

Can parse any context-free grammar

Closed under union of languages

Support for production priority

Tool easily adapts to other parsing algorithms


Unparsing:Canonical whitespace

Ambiguity: parsingunparsingAmbiguity: parsingunparsing

ASTL / ~L

.

L

. Parsing:Grammar ambiguity

ASTL / ~L

.

L

.

.


Ambiguity AnalysisAmbiguity Analysis

Ambiguity Analysis:

Using implementation ( ) on:Source language;

Target language; and/or

…all intermediate languages (somewhat expensive)

(Note: Ambiguity analysis comes with XSugar tool)

"Analyzing Ambiguity of Context-Free Grammars" [ Claus Brabrand | Robert Giegerich | Anders Møller ] ( CIAA 2007 )

"dk.brics.grammar"[ by Anders Møller ]


Error ReportingError Reporting

Error reporting:Static parse-error (O'Caml-lex):

Static transformation error (XSugar):

(is actually a parse-error in a cata reconstructor)

Dynamic parse-error (XSugar):

Dynamic transformation error:impossible :-)

*** In ln2l.x (4,4)-(4,7): Parse error at "Exp"

*** Parse error at character 6 (line 1, column 7) in /tmp/shape84e645.txt

*** Parse error at character 23 (line 1, column 24) in /dev/stdin

Could beimproved

Prototype

[ 1 ] claus brabrand, itu banana algebra diku, ku, denmark may 11, 2010 banana algebra: jacob...

Documents