talk lund university cs department

Modularity and Composition of LanguageSpecifications

Ted Kaminski, Lijesh Krishnan, Yogesh Mali, AugustSchwerdfeger and Eric Van Wyk

University of Minnesota

September 17, 2012, Lund, Sweden

Thanks to the National Science Foundation (NSF), Fundacao Luso-Americana(FLAD), McKnight Foundation, IBM, and the University of Minnsota for fundingportions of this work.

Extensible Language Frameworks — ableP

I add features to a “host” language — PromelaI new language constructs - their syntax and semantics

I select (altitude: 1000 .. 10000);I select (altitude: 1000 .. 10000 step 100);I select (altQuality: High, Med, Low);I DTSpin constructs: timer t; t = 1; expire(t);

I new semantics of existing constructssemantic analysis, translations to new target languages, ...

I type checkingI advanced ETCH-style type inference and checking

Various means for extending Promela

I select (v: 1 .. 10) added in SPIN version 6.I DTSPIN features

I as CPP macros — lightweightI or modifying the SPIN implementation — heavyweight

I ETCH, enhanced type checkingI built their own scanner and parser using SableCC

I ableP — middleweight approach

An example

An altitude switch model that uses

I enhanced select statements

I DTSPIN-like constructs

I tabular Boolean expressions (a la RSML and SCR)

An instance of ableP parses and analyzes the model, thengenerates its translation to pure Promela.

% java -jar ableP.aviation.jar AltSwitch.xpml

% spin -a AltSwitch.pml

Our approach:I Users choose (independently developed) extensions.

I Tools compose the extensions and Promela host language.

I Distinguish1. extension user

I has no knowledge of language design or implementations

2. extension developerI must know about language design and implementation

1. Tools and formalisms support automatic composition.

2. Modular analyses ensure the composition results in a workingtranslator.

I Value easy composition over expressivity, accept somerestrictions

I on syntaxI new constructs are translated to “pure” Promela

I ableP “instances” are smart pre-processors

Extending ableP with independently developed extensions

I Extension user directs underlying tools toI compose chosen extensions and the host language

I to create a custom translator for the extended language

I Silver grammar modules define sets of specificationsI composition is set union, order does not matter

I Consider the Silver specification for this composition.

Developing language extensions

Two primary challenges:

1. composable syntax — enables building a parserI context-aware scanning [GPCE’07]I modular determinism analysis [PLDI’09]I Copper

2. composable semantics — analysis and translations

I attribute grammars with forwarding, collections andhigher-order attributes

I set union of specification componentsI sets of productions, non-terminals, attributesI sets of attribute defining equations, on a productionI sets of equations contributing values to a single attribute

I modular well-definedness analysis [SLE’12a]I monolithic termination analysis [SLE’12b]I Silver

Context aware scanning

I Scanner recognizes only tokens valid for current “context”

I keeps embedded sub-languages, in a sense, separateI Consider:

I chan in, out;

for i in a { a[i] = i*i ; }

I Two terminal symbols that match “in”.I terminal IN ’in’ ;I terminal ID /[a-zA-Z ][a-zA-Z 0-9]*/

submits to {promela kwd };

I terminal FOR ’for’ lexer classes {promela kwd };

Allows parsing of embedded C code

c_decl {

typedef struct Coord {

int x, y; } Coord; }

c_state "Coord pt" "Global" /* goes in state vector */

int z = 3; /* standard global decl */

active proctype example()

{ c_code { now.pt.x = now.pt.y = 0; };

do :: c_expr { now.pt.x == now.pt.y }

-> c_code { now.pt.y++; }

:: else -> break

od;

c_code { printf("values %d: %d, %d,%d\n",

Pexample->_pid, now.z, now.pt.x, now.pt.y);

};

}

// assert(false) /* trigger an error trail */

//}

Semantics for host language assignment constructs

grammar edu:umn:cs:melt:ableP:host:core:abstractsyntax;

abstract production defaultAssign

s::Stmt ::= lhs::Expr rhs::Expr

{ s.pp = lhs.pp ++ " = " ++ rhs.pp ++ " ;\n" ;

lhs.env = s.env; rhs.env = s.env;

s.defs = emptyDefs();

s.errors := lhs.errors ++ rhs.errors ;

}

Adding extension constructs involves writing similar productions.

Adding ETCH-like semantic analysis.

grammar edu:umn:cs:melt:ableP:extensions:typeChecking ;

synthesized attribute typerep::TypeRep

occurs on Expr, Decls ;

aspect production varRef

e::Expr ::= id::ID

{ e.typerep = ... retrieve from declaration

found in e.env ... ; }

aspect production defaultAssign

s::Stmt ::= lhs::Expr rhs::Expr

{ s.errors <- if isCompatible(lhs.typerep, rhs.typerep)

then [ ]

else [ mkError ("Incompatible types ...") ];

}

Extensibility: safe composability

Host

Ext 1

Ext 2

Problem

New attributesN

ew

pro

duct

ions

independentextensions

Extensibility: safe composability

Extensions get undefined semantics from host translation.

grammar edu:umn:cs:melt:ableP:extensions:enhancedSelect ;

abstract production selectFrom

s::Stmt ::= sl::’select’ v::Expr es::Exprs

{

s.pp = "select ( " ++ v.pp ++ ":" ++ es.pp ++ " ); \n" ;

s.errors := v.errors ++ es.errors ++

if ... check that all expressions in ’es’ have

same type as ’v’ ...

then [ mkError ("Error: select statement " ++

"requires same type ... ") ]

else [ ] ;

forwards to ifStmt( mkOptions (v, es) ) ;

}

Modular analysis

Ensuring that the composition will be successful.

Context free grammars

GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE

I ∪ of sets of nonterminals, terminals, productions

I Composition of all is an context free grammar.

I Is it non-ambiguous, useful for deterministic (LR) parsing?

I conflictFree(GH ∪ G 1E ) holds

I conflictFree(GH ∪ G 2E ) holds

I conflictFree(GH ∪ G iE ) holds

I conflictFree(GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE ) may not hold

Attribute grammars

AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE

I ∪ of sets of attributes, attribute equations, occurs-ondeclarations

I Composition of all is an attribute grammar.

I Completeness: ∀ production, ∀ attribute, ∃ an equation

I complete(AGH ∪ AG 1E ) holds

I complete(AGH ∪ AG 2E ) holds

I complete(AGH ∪ AG iE ) holds

I complete(AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE ) may not hold

I similarly for non-circularity of the AG

Detecting problems, ensuring composition

When can some analysis of the language specification be applied?When ...

1. the host language is developed ?

2. a language extensions is developed ?

3. when the host and extensions are composed ?

4. when the resulting language tools are run ?

Libraries, and modular type checking

I Libraries “just work”

I Type checking is done by the library writer, modularly.

I Language extensions should be like libraries, composition of“verified” extensions should “just work.”

Modular determinism analysis for grammars, 2009

GH ∪ G 1E ∪ G 2

E ∪ ... ∪ G iE

I isComposable(GH ,G 1E ) ∧ conflictFree(GH ∪ G 1

E ) holds

I isComposable(GH ,G 2E ) ∧ conflictFree(GH ∪ G 2

E ) holds

I isComposable(GH ,G iE ) ∧ conflictFree(GH ∪ G i

E ) holds

I these imply conflictFree(GH ∪ G 1E ∪ G 2

E ∪ ...) holds

I ( ∀i ∈ [1, n].isComposable(GH ,G iE ) ∧

conflictFree(GH ∪ {G iE )} )

=⇒ conflictFree(GH ∪ {G 1E , . . . ,G n

E})I Some restrictions to extension introduced syntax apply, of

course.

Modular completeness analysis for attribute grammars

AGH ∪ AG 1E ∪ AG 2

E ∪ ... ∪ AG iE

I modComplete(AGH ∪ AG 1E ) holds

I modComplete(AGH ∪ AG 2E ) holds

I modComplete(AGH ∪ AG iE ) holds

I these imply complete(AGH ∪ AG 1E ∪ AG 2

E ∪ ...) holds

I (∀i ∈ [1, n].modComplete(AGH ,AG iE ))

=⇒ complete(AGH ∪ {AG 1E , ...,AG n

E}).I similarly for non-circularity of the AG

I Again, some restrictions on extensions.

Modular completeness analysis

I The analysis modComplete is defined as follows:

modComplete(AGH ,AGE ) ,noOrphanOccursOn(AGH ,AGE ) ∧noOrphanAttrEqs(AGH ,AGE ) ∧noOrphanProds(AGH ,AGE ) ∧synComplete(AGH ,AGE ) ∧modularFlowTypes(flowTypes(AGH),

flowTypes(AGH ∪ AGE )) ∧inhComplete(AGH ,AGE , flowTypes(AGH ∪ AGE ))

I Some “structural” requirements, some flow-type requirements

I Silver’s module system prevents duplicate nonterminal,attribute, production declarations.

No orphan occurs on declarations

“an AG declares a occurs on nt only if it declares or exports a

or nt”

noOrphanOccursOn(AGH ,AGE )

holds if and only if each occurs-on declaration“attribute a occurs on nt” in AGH ∪ AGE

is exported by the grammar declaring a or the grammardeclaring nt.

No orphan attribute equations

“an AG provides an equation n.a=e on p with l.h.s. nonterminalnt only if it declares/exports the production p or the occurs-ondeclaration a occurs on nt.”

noOrphanAttrEqs(AGH ,AGE )

holds if and only if each equation n.a = e in a production p isexported by the grammar declaring the (non-aspect)production p or the grammar declaring the occurs-ondeclaration “attribute a occurs on nt” (where n has typent.)

No orphan production declarations

“Productions in extensions that build host language nonterminalsmust forward.”

noOrphanProds(AGH ,AGE )

holds if and only if for each production declaration p inAGH ∪ AGE with left hand side nonterminal nt, theproduction p is either exported by the grammar declaring nt,or p forwards.

Completeness of synthesized equations

“A production forwards or has equations for all its attributes, thesemay be on aspect productions.”

synComplete(AGH ,AGE )

holds if and only if for each occurs-on declarationattribute a occurs on nt, and for each non-forwardingproduction p that constructs nt, there exists a rule definingthe synthesized equation p : x .a, where x is the left hand sideof the production.

Flow Types

I A flow-type captures how information flow between attributes.

I For a non-terminal, it maps synthesized attributes to theinherited attributes on which it depends.

I ftnt :: As → 2AI

I e.g. ftExpr (type) = {env}

Modularity of flow types

“Host language attributes on host language nonterminals do notdepend on extension-declared inherited attributes.”

modularFlowTypes(flowTypes(AGH),

flowTypes(AGH ∪ AGE ))

holds if and only if for each ftHnt ∈ flowTypes(AGH) andftH∪Ent ∈ flowTypes(AGH ∪ AGE ),

for all synthesized attributes s and for all nonterminals nt suchthat attribute s occurs on nt is declared in AGH ,ftH∪Ent (s) ⊆ ftHnt(s).

Effective completeness of inherited equations

“All inherited attributes required to compute a synthesizedattribute on a node have defining equations.”

inhComplete(AGH ,AGE , flowTypes(AGH ∪ AGE ))

holds if and only if for every production p in AGH ∪ AGE andfor every access to a synthesized attribute n.s in an expressionwithin p (where n has type nt,) and for each inheritedattribute i ∈ ftnt(s), there exists an equation n.i = e for p.

A generalization

I Silver does not identify “host” and “extensions.”

I The analysis works on import grammar relationships.

I A grammar A that imports B is seen as an extension to B.

I So, Silver libraries are hosts, in essence.

Evaluation

I Q: Are these restrictions too overbearing?A1: No, they are turned on by default.A2: No, except for reference attributes.

I Applied to Silver compiler’s Silver specs

1. We found a few bugs.2. We moved some declarations to new grammars.

These were bad design “smells.”3. We extended Silver

I Annotations allowing host language to be modularized withoutrespect to the rules.These treat the “whole” host language, spread across manymodules, as one for the analysis.

I Default attributes.

I Reference attributes - rather severe restrictions.All inherited attributes must be provided and no more can beadded.

Modular circularity analysis

I Analysis extends to check for no cycles, modularly.

I Instead of a flow type for each non-terminal, there is a set offlow-graphs for each.

I Analysis ensures no new patterns of information flow areadded by an extension.

Lessons learned ?

I For extensible language frameworksI Analysis - modular or don’t bother?

I But my undergrads like static completeness detection.

Termination analysis for Higher Order AGs

I detects potential creation of an infinite number of trees.

I also a restricted modular version.

This, with modular well-definedness analysis, ensures that thecomposed AG will terminate normally.

So ... for ableP

I ableP supports the simple composition of language extensions

I This creates translators and analyzers for customizedPromela-based languages.

I extensions can be verified to compose, with other verifiedextensions — done by extension developers

I adding (independently developed) extensions that add newfeatures and new analysis on host features is supported

Some questions ... what to call these?

I We’ve called this a “modular” analysis.

I Maybe “static composition analysis”I analysis that happens before composition

I As opposed to “dynamic composition analysis”I analysis that happens during composition

I But “static” and “dynamic” suggest 2 points in timeI before and during run-time

I We have more than 2 interesting points in time.I 1. host development, 2. extension development,

3. composition, 4. translation, 5. execution

Future work

I These modular analyses, for syntax and static semantics,ensure the generated translator terminates normally.

I They don’t ensure that the translator is correct.

I Host language specific invariants are not enforced.

I How can we do that?

Thanks for your attention.

Questions?

http://melt.cs.umn.edu/

[email protected]

http://melt.cs.umn.edu/

[email protected]

[GPCE’07] Eric Van Wyk and August Schwerdfeger.Context-aware scanning for parsing extensiblelanguages.In Intl. Conf. on Generative Programming andComponent Engineering, (GPCE). ACM Press,October 2007.

[PLDI’09] August Schwerdfeger and Eric Van Wyk.Verifiable composition of deterministic grammars.In Proc. of ACM SIGPLAN Conference onProgramming Language Design and Implementation(PLDI). ACM Press, June 2009.

[SLE’12a] Ted Kaminski and Eric Van Wyk.Modular well-definedness analysis for attributegrammars.In Proceedings of 5th the International Conferenceon Software Language Engineering (SLE 2012),LNCS. Springer-Verlag, September 2012.To appear.

[SLE’12b] Lijesh Krishnan and Eric Van Wyk.Termination analysis for higher-order attributegrammars.In Proceedings of 5th the International Conferenceon Software Language Engineering (SLE 2012),LNCS. Springer-Verlag, September 2012.To appear.

talk lund university cs department

Documents