talk lund university cs department

Modularity and Composition of LanguageSpecifications

Ted Kaminski, Lijesh Krishnan, Yogesh Mali, AugustSchwerdfeger and Eric Van Wyk

University of Minnesota

September 17, 2012, Lund, Sweden

Thanks to the National Science Foundation (NSF), Fundacao Luso-Americana(FLAD), McKnight Foundation, IBM, and the University of Minnsota for fundingportions of this work.

Page 1

Extensible Language Frameworks — ableP

I add features to a “host” language — PromelaI new language constructs - their syntax and semantics

I select (altitude: 1000 .. 10000);I select (altitude: 1000 .. 10000 step 100);I select (altQuality: High, Med, Low);I DTSpin constructs: timer t; t = 1; expire(t);

I new semantics of existing constructssemantic analysis, translations to new target languages, ...

I type checkingI advanced ETCH-style type inference and checking

Page 2

Various means for extending Promela

I select (v: 1 .. 10) added in SPIN version 6.I DTSPIN features

I as CPP macros — lightweightI or modifying the SPIN implementation — heavyweight

I ETCH, enhanced type checkingI built their own scanner and parser using SableCC

I ableP — middleweight approach

Page 3

An example

An altitude switch model that uses

I enhanced select statements

I DTSPIN-like constructs

I tabular Boolean expressions (a la RSML and SCR)

An instance of ableP parses and analyzes the model, thengenerates its translation to pure Promela.

% java -jar ableP.aviation.jar AltSwitch.xpml

% spin -a AltSwitch.pml

Page 4

Our approach:I Users choose (independently developed) extensions.

I Tools compose the extensions and Promela host language.

I Distinguish1. extension user

I has no knowledge of language design or implementations

2. extension developerI must know about language design and implementation

1. Tools and formalisms support automatic composition.

2. Modular analyses ensure the composition results in a workingtranslator.

I Value easy composition over expressivity, accept somerestrictions

I on syntaxI new constructs are translated to “pure” Promela

I ableP “instances” are smart pre-processors

Page 5

Extending ableP with independently developed extensions

I Extension user directs underlying tools toI compose chosen extensions and the host language

I to create a custom translator for the extended language

I Silver grammar modules define sets of specificationsI composition is set union, order does not matter

I Consider the Silver specification for this composition.

Page 6

Developing language extensions

Two primary challenges:

1. composable syntax — enables building a parserI context-aware scanning [GPCE’07]I modular determinism analysis [PLDI’09]I Copper

2. composable semantics — analysis and translations

I attribute grammars with forwarding, collections andhigher-order attributes

I set union of specification componentsI sets of productions, non-terminals, attributesI sets of attribute defining equations, on a productionI sets of equations contributing values to a single attribute

I modular well-definedness analysis [SLE’12a]I monolithic termination analysis [SLE’12b]I Silver

Page 7

Context aware scanning

I Scanner recognizes only tokens valid for current “context”

I keeps embedded sub-languages, in a sense, separateI Consider:

I chan in, out;

for i in a { a[i] = i*i ; }

I Two terminal symbols that match “in”.I terminal IN ’in’ ;I terminal ID /[a-zA-Z ][a-zA-Z 0-9]*/

submits to {promela kwd };

I terminal FOR ’for’ lexer classes {promela kwd };

Page 8

Allows parsing of embedded C code

c_decl {

typedef struct Coord {

int x, y; } Coord; }

c_state "Coord pt" "Global" /* goes in state vector */

int z = 3; /* standard global decl */

active proctype example()

{ c_code { now.pt.x = now.pt.y = 0; };

do :: c_expr { now.pt.x == now.pt.y }

-> c_code { now.pt.y++; }

:: else -> break

od;

c_code { printf("values %d: %d, %d,%d\n",

Pexample->_pid, now.z, now.pt.x, now.pt.y);

};

}

// assert(false) /* trigger an error trail */

//}

Page 9

Semantics for host language assignment constructs

grammar edu:umn:cs:melt:ableP:host:core:abstractsyntax;

abstract production defaultAssign

s::Stmt ::= lhs::Expr rhs::Expr

{ s.pp = lhs.pp ++ " = " ++ rhs.pp ++ " ;\n" ;

lhs.env = s.env; rhs.env = s.env;

s.defs = emptyDefs();

s.errors := lhs.errors ++ rhs.errors ;

}

Adding extension constructs involves writing similar productions.

Page 10

Adding ETCH-like semantic analysis.

grammar edu:umn:cs:melt:ableP:extensions:typeChecking ;

synthesized attribute typerep::TypeRep

occurs on Expr, Decls ;

aspect production varRef

e::Expr ::= id::ID

{ e.typerep = ... retrieve from declaration

found in e.env ... ; }

aspect production defaultAssign

s::Stmt ::= lhs::Expr rhs::Expr

{ s.errors <- if isCompatible(lhs.typerep, rhs.typerep)

then [ ]

else [ mkError ("Incompatible types ...") ];

}

Page 11

Extensibility: safe composability

Host

Ext 1

Ext 2

Problem

New attributesN

ew

pro

duct

ions

independentextensions

Page 12

Extensibility: safe composability

Page 13

Extensions get undefined semantics from host translation.

grammar edu:umn:cs:melt:ableP:extensions:enhancedSelect ;

abstract production selectFrom

s::Stmt ::= sl::’select’ v::Expr es::Exprs

{

s.pp = "select ( " ++ v.pp ++ ":" ++ es.pp ++ " ); \n" ;

s.errors := v.errors ++ es.errors ++

if ... check that all expressions in ’es’ have

same type as ’v’ ...

then [ mkError ("Error: select statement " ++

"requires same type ... ") ]

else [ ] ;

forwards to ifStmt( mkOptions (v, es) ) ;

}

Page 14

Modular analysis

Ensuring that the composition will be successful.

Page 15

Context free grammars

GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE

I ∪ of sets of nonterminals, terminals, productions

I Composition of all is an context free grammar.

I Is it non-ambiguous, useful for deterministic (LR) parsing?

I conflictFree(GH ∪ G 1E ) holds

I conflictFree(GH ∪ G 2E ) holds

I conflictFree(GH ∪ G iE ) holds

I conflictFree(GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE ) may not hold

Page 16

Attribute grammars

AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE

I ∪ of sets of attributes, attribute equations, occurs-ondeclarations

I Composition of all is an attribute grammar.

I Completeness: ∀ production, ∀ attribute, ∃ an equation

I complete(AGH ∪ AG 1E ) holds

I complete(AGH ∪ AG 2E ) holds

I complete(AGH ∪ AG iE ) holds

I complete(AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE ) may not hold

I similarly for non-circularity of the AG

Page 17

Detecting problems, ensuring composition

When can some analysis of the language specification be applied?When ...

1. the host language is developed ?

2. a language extensions is developed ?

3. when the host and extensions are composed ?

4. when the resulting language tools are run ?

Page 18

Libraries, and modular type checking

I Libraries “just work”

I Type checking is done by the library writer, modularly.

I Language extensions should be like libraries, composition of“verified” extensions should “just work.”

Page 19

Modular determinism analysis for grammars, 2009

GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE

I isComposable(GH ,G 1E ) ∧ conflictFree(GH ∪ G 1

E ) holds

I isComposable(GH ,G 2E ) ∧ conflictFree(GH ∪ G 2

E ) holds

I isComposable(GH ,G iE ) ∧ conflictFree(GH ∪ G i

E ) holds

I these imply conflictFree(GH ∪ G 1E ∪ G 2

E ∪ ...) holds

I ( ∀i ∈ [1, n].isComposable(GH ,G iE ) ∧

conflictFree(GH ∪ {G iE )} )

=⇒ conflictFree(GH ∪ {G 1E , . . . ,G n

E})I Some restrictions to extension introduced syntax apply, of

course.

Page 20

Modular completeness analysis for attribute grammars

AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE

I modComplete(AGH ∪ AG 1E ) holds

I modComplete(AGH ∪ AG 2E ) holds

I modComplete(AGH ∪ AG iE ) holds

I these imply complete(AGH ∪ AG 1E ∪ AG 2

E ∪ ...) holds

I (∀i ∈ [1, n].modComplete(AGH ,AG iE ))

=⇒ complete(AGH ∪ {AG 1E , ...,AG n

E}).I similarly for non-circularity of the AG

I Again, some restrictions on extensions.

Page 21

Modular completeness analysis

I The analysis modComplete is defined as follows:

modComplete(AGH ,AGE ) ,noOrphanOccursOn(AGH ,AGE ) ∧noOrphanAttrEqs(AGH ,AGE ) ∧noOrphanProds(AGH ,AGE ) ∧synComplete(AGH ,AGE ) ∧modularFlowTypes(flowTypes(AGH),

flowTypes(AGH ∪ AGE )) ∧inhComplete(AGH ,AGE , flowTypes(AGH ∪ AGE ))

I Some “structural” requirements, some flow-type requirements

I Silver’s module system prevents duplicate nonterminal,attribute, production declarations.

Page 22

No orphan occurs on declarations

“an AG declares a occurs on nt only if it declares or exports a

or nt”

noOrphanOccursOn(AGH ,AGE )

holds if and only if each occurs-on declaration“attribute a occurs on nt” in AGH ∪ AGE

is exported by the grammar declaring a or the grammardeclaring nt.

Page 23

No orphan attribute equations

“an AG provides an equation n.a=e on p with l.h.s. nonterminalnt only if it declares/exports the production p or the occurs-ondeclaration a occurs on nt.”

noOrphanAttrEqs(AGH ,AGE )

holds if and only if each equation n.a = e in a production p isexported by the grammar declaring the (non-aspect)production p or the grammar declaring the occurs-ondeclaration “attribute a occurs on nt” (where n has typent.)

Page 24

No orphan production declarations

“Productions in extensions that build host language nonterminalsmust forward.”

noOrphanProds(AGH ,AGE )

holds if and only if for each production declaration p inAGH ∪ AGE with left hand side nonterminal nt, theproduction p is either exported by the grammar declaring nt,or p forwards.

Page 25

Completeness of synthesized equations

“A production forwards or has equations for all its attributes, thesemay be on aspect productions.”

synComplete(AGH ,AGE )

holds if and only if for each occurs-on declarationattribute a occurs on nt, and for each non-forwardingproduction p that constructs nt, there exists a rule definingthe synthesized equation p : x .a, where x is the left hand sideof the production.

Page 26

Flow Types

I A flow-type captures how information flow between attributes.

I For a non-terminal, it maps synthesized attributes to theinherited attributes on which it depends.

I ftnt :: As → 2AI

I e.g. ftExpr (type) = {env}

Page 27

Modularity of flow types

“Host language attributes on host language nonterminals do notdepend on extension-declared inherited attributes.”

modularFlowTypes(flowTypes(AGH),

flowTypes(AGH ∪ AGE ))

holds if and only if for each ftHnt ∈ flowTypes(AGH) andftH∪Ent ∈ flowTypes(AGH ∪ AGE ),

for all synthesized attributes s and for all nonterminals nt suchthat attribute s occurs on nt is declared in AGH ,ftH∪Ent (s) ⊆ ftHnt(s).

Page 28

Effective completeness of inherited equations

“All inherited attributes required to compute a synthesizedattribute on a node have defining equations.”

inhComplete(AGH ,AGE , flowTypes(AGH ∪ AGE ))

holds if and only if for every production p in AGH ∪ AGE andfor every access to a synthesized attribute n.s in an expressionwithin p (where n has type nt,) and for each inheritedattribute i ∈ ftnt(s), there exists an equation n.i = e for p.

Page 29

A generalization

I Silver does not identify “host” and “extensions.”

I The analysis works on import grammar relationships.

I A grammar A that imports B is seen as an extension to B.

I So, Silver libraries are hosts, in essence.

Page 30

Evaluation

I Q: Are these restrictions too overbearing?A1: No, they are turned on by default.A2: No, except for reference attributes.

I Applied to Silver compiler’s Silver specs

1. We found a few bugs.2. We moved some declarations to new grammars.

These were bad design “smells.”3. We extended Silver

I Annotations allowing host language to be modularized withoutrespect to the rules.These treat the “whole” host language, spread across manymodules, as one for the analysis.

I Default attributes.

I Reference attributes - rather severe restrictions.All inherited attributes must be provided and no more can beadded.

Page 31

Modular circularity analysis

I Analysis extends to check for no cycles, modularly.

I Instead of a flow type for each non-terminal, there is a set offlow-graphs for each.

I Analysis ensures no new patterns of information flow areadded by an extension.

Page 32

Lessons learned ?

I For extensible language frameworksI Analysis - modular or don’t bother?

I But my undergrads like static completeness detection.

Page 33

Termination analysis for Higher Order AGs

I detects potential creation of an infinite number of trees.

I also a restricted modular version.

This, with modular well-definedness analysis, ensures that thecomposed AG will terminate normally.

Page 34

So ... for ableP

I ableP supports the simple composition of language extensions

I This creates translators and analyzers for customizedPromela-based languages.

I extensions can be verified to compose, with other verifiedextensions — done by extension developers

I adding (independently developed) extensions that add newfeatures and new analysis on host features is supported

Page 35

Some questions ... what to call these?

I We’ve called this a “modular” analysis.

I Maybe “static composition analysis”I analysis that happens before composition

I As opposed to “dynamic composition analysis”I analysis that happens during composition

I But “static” and “dynamic” suggest 2 points in timeI before and during run-time

I We have more than 2 interesting points in time.I 1. host development, 2. extension development,

3. composition, 4. translation, 5. execution

Page 36

Future work

I These modular analyses, for syntax and static semantics,ensure the generated translator terminates normally.

I They don’t ensure that the translator is correct.

I Host language specific invariants are not enforced.

I How can we do that?

Page 37

Thanks for your attention.

Questions?

http://melt.cs.umn.edu/

evw@cs.umn.edu

Page 38

[GPCE’07] Eric Van Wyk and August Schwerdfeger.Context-aware scanning for parsing extensiblelanguages.In Intl. Conf. on Generative Programming andComponent Engineering, (GPCE). ACM Press,October 2007.

[PLDI’09] August Schwerdfeger and Eric Van Wyk.Verifiable composition of deterministic grammars.In Proc. of ACM SIGPLAN Conference onProgramming Language Design and Implementation(PLDI). ACM Press, June 2009.

Page 39

[SLE’12a] Ted Kaminski and Eric Van Wyk.Modular well-definedness analysis for attributegrammars.In Proceedings of 5th the International Conferenceon Software Language Engineering (SLE 2012),LNCS. Springer-Verlag, September 2012.To appear.

[SLE’12b] Lijesh Krishnan and Eric Van Wyk.Termination analysis for higher-order attributegrammars.In Proceedings of 5th the International Conferenceon Software Language Engineering (SLE 2012),LNCS. Springer-Verlag, September 2012.To appear.

Page 40