Abstract Syntax

Cheng-Chia Chen

Cheng-Chia Chen 2

Concrete v.s. Abstract Syntax

• Parsing results can generally be represented in two forms:– concrete syntax tree (CST or Parse Tree)

– abstract syntax tree (AST or syntax tree)

• Concrete syntax trees (parse tree) is a record of the rules (and tokens) used to match some input text.

• Abstract syntax tree records the structure of the input and is insensitive to the grammar that produced it.

• AST is superior to CST since– CST reflects the structure of a sentence in a grammar

• AST reflects the structure of a sentence in the language.• But a language can have a lot of grammars.

– AST is generally more compact.– AST is usually more convenient for later analysis and

processing.

Cheng-Chia Chen 3

Example

• Given the grammar :

Expr Expr + Term

Term Term * Factor

Factor Number | (E)

• and the input ‘2 + 5 ‘ :• We should have the following CST and AST:

Cheng-Chia Chen 4

PlusExpr

NumExpr:5

NumExpr:2

e1 e2

BinExprop: plus

NumExpr:5

NumExpr:2

e1 e2

Expr

Factor

Plus :+

Term

Number:2

Term

Factor

Number :5

Expr

The parse tree for ‘2+5’Possible ASTs for ‘2+5’

Cheng-Chia Chen 5

Abstract Syntax v.s. Concrete Syntax of ExpressionsAbstract Syntax • E E + E• E E – E• E E * E • E E / E• E id • E num Notes:1. reflect the meaning of

expressions; resulting AST more convenient for later processing

2. ambiguous; impractical to parsing

Concrete Syntax (LALR(1))• E E + F• E E – F• F F * T • F T / T• T id • T num• T (E)

Notes:

1. quick parse possible

2. resulting CST is highly redundant

Conclusion: generate AST (instead of CST) during concrete syntax parsing

Cheng-Chia Chen 6

Building Syntax Tree for Expressions(using JavaCC)

• First devise the Concrete Syntax (LL(K)) in EBNF Form.

Start Exp

Exp Term ( “+” Term | “-” Tem)*

Term Factor (“*” Factor | “/” Factor)*

Factor <IDENTIFIER> | <INTEGER_LITERAL>

| “(“ Exp “)”

Cheng-Chia Chen 7

Design the AST classes : Class Hierarchy for expressions• Exp• PlusExp• MinusExp• TimesExp• DivideExp• Identifier• IntergerLiteral

or simply a BinaryExp

Cheng-Chia Chen 8

AST classes for expressions

pubic abstract class Exp { pubic abstract int eval() }public class PlusExp extends Exp { private Exp e1,e2 ; public PlusExp(Exp a1, Exp a2) { e1=a1;e2=a2;} public int eval() { return e1.eval() + e2.eval(); }}…/* similar to PlusExp for class MinusExp, timesExp and DivideExp. Omitted here */public class Identifier extends Exp { pubic String f0;public Identifier(String a0) { f0 = a0 ; }public int eval() { return symbolTable.lookup(f0); }}

Cheng-Chia Chen 9

AST classes

public class IntergerLiteral extends Exp {

pubic String f0;

public Identifier(String a0) { f0 = a0 ; }

public int eval() { return Integer.parseInt(f0); }

}

Cheng-Chia Chen 10

Add actions to build syntax tree :Using the available AST classes

Exp Start() : {Exp e;}

{ e = Exp() { return e;}

}

Exp Exp() : {Exp e1,e2;}

{ e1 = Term()

( “+” e2 = Term() { e1 = new PlusExp(e1,e2) ;}

| “-” e2 = Term() { e1 = new MinusExp(e1,e2) ;}

)*

{ return e1; }

}

Cheng-Chia Chen 11

Building AST

Exp Term() : {Exp e1,e2;}{ e1 = Factor() ( “*” e2 = Factor() { e1 = new TimesExp(e1,e2) ;} | “/” e2 = Factor() { e1 = new DivideExp(e1,e2) ;} )* { return e1; }}

Exp Factor() : {Token t; Exp e;}{ ( t = <IDENTIFIER> { e = new Identifier(t.image); } | t= <INTEGER_LITERAL> { e = new IntegerLiteral(t.image); } | “(“ e = Exp() “)” ) { return e; }}

Cheng-Chia Chen 12

ExpParser.jjoptions { STATIC = false; … }PARSER_BEGIN(ExpParser) public class ExpParser { … public static void main( String[] args) { ExpParser p = new ExpParser(System.in); Exp ast = p.start(); System.out.prinln( “result = “ + ast.eval() ); }Token : {<IDENTIFIER: … ><INTEGER_LITERAL: …>}SKIP : { … }…

Cheng-Chia Chen 13

• Tools for automated generation of AST classes definitions from abstract syntax definition– sableCC version 3.X– JTB

• Disadvantage– The syntax tree and relevant classes generated may be less

abstract than one could desire.

Cheng-Chia Chen 14

ASTs for MiniJava

• at package syntaxtree;

Cheng-Chia Chen 15

Abstract Syntax for MiniJava (I)

Package syntaxtree;

Program(MainClass m, ClassDecList c1)MainClass(Identifier i1, Identifier i2, Statement s)----------------------------abstract class ClassDeclClassDeclSimple(Identifier i, VarDeclList vl, methodDeclList m1)ClassDeclExtends(Identifier i, Identifier j, VarDecList vl, MethodDeclList ml)-----------------------------VarDecl(Type t, Identifier i)MethodDecl(Type t, Identifier I, FormalList fl, VariableDeclList vl, StatementList sl, Exp e)Formal(Type t, Identifier i)

Cheng-Chia Chen 16

Abstract Syntax for MiniJava (II)

abstract class typeIntArrayType()BooleanType()IntegerType()IndentifierType(String s)---------------------------abstract class StatementBlock(StatementList sl)If(Exp e, Statement s1, Statement s2)While(Exp e, Statement s)Print(Exp e)Assign(Identifier i, Exp e)ArrayAssign(Identifier i, Exp e1, Exp e2)-------------------------------------------

Cheng-Chia Chen 17

Abstract Syntax for MiniJava (III)abstract class ExpAnd(Exp e1, Exp e2) LessThan(Exp e1, Exp e2)Plus(Exp e1, Exp e2) Minus(Exp e1, Exp e2)Times(Exp e1, Exp e2) Not(Exp e)ArrayLookup(Exp e1, Exp e2) ArrayLength(Exp e)Call(Exp e, Identifier i, ExpList el)IntegerLiteral(int i)True() False()IdentifierExp(String s)This()NewArray(Exp e) NewObject(Identifier i)-------------------------------------------------Identifier(Sting s)--list classes-------------------------ClassDecList() ExpList() FormalList() MethodDeclList()StatementLIst() VarDeclList()

Cheng-Chia Chen 18

Syntax Tree Nodes - Detailspackage syntaxtree;import visitor.Visitor;import visitor.TypeVisitor;

public class Program { public MainClass m; public ClassDeclList cl;

public Program(MainClass am, ClassDeclList acl) { m=am; cl=acl; }

public void accept(Visitor v) { v.visit(this); }

public Type accept(TypeVisitor v) { return v.visit(this); }}

Cheng-Chia Chen 19

ClassDecl.java

package syntaxtree;

import visitor.Visitor;

import visitor.TypeVisitor;

public abstract class ClassDecl {

public abstract void accept(Visitor v);

public abstract Type accept(TypeVisitor v);

}

Cheng-Chia Chen 20

ClassDeclExtends.java

package syntaxtree;import visitor.Visitor;import visitor.TypeVisitor;

public class ClassDeclExtends extends ClassDecl { public Identifier i; public Identifier j; public VarDeclList vl; public MethodDeclList ml; public ClassDeclExtends(Identifier ai, Identifier aj, VarDeclList avl, MethodDeclList aml) { i=ai; j=aj; vl=avl; ml=aml; } public void accept(Visitor v) { v.visit(this); } public Type accept(TypeVisitor v) { return v.visit(this); }}

Cheng-Chia Chen 21

StatementList.java

package syntaxtree;import java.util.Vector;

public class StatementList { private Vector list;

public StatementList() { list = new Vector(); } public void addElement(Statement n) { list.addElement(n); } public Statement elementAt(int i) { return (Statement)list.elementAt(i); } public int size() { return list.size(); }}

Cheng-Chia Chen 22

Package visitor/Visitor.javapackage visitor;import syntaxtree.*;

public interface Visitor { public void visit(Program n); public void visit(MainClass n); public void visit(ClassDeclSimple n); public void visit(ClassDeclExtends n); public void visit(VarDecl n); public void visit(MethodDecl n); public void visit(Formal n); public void visit(IntArrayType n); public void visit(BooleanType n); public void visit(IntegerType n); public void visit(IdentifierType n); public void visit(Block n); public void visit(If n); public void visit(While n); public void visit(Print n); public void visit(Assign n); public void visit(ArrayAssign n); public void visit(And n); public void visit(LessThan n); public void visit(Plus n); public void visit(Minus n); public void visit(Times n); public void visit(ArrayLookup n); public void visit(ArrayLength n); public void visit(Call n); public void visit(IntegerLiteral n); public void visit(True n); public void visit(False n); public void visit(IdentifierExp n); public void visit(This n); public void visit(NewArray n); public void visit(NewObject n); public void visit(Not n); public void visit(Identifier n);}

Cheng-Chia Chen 23

AST for statement : X = y.m(1,4+5)

Statement s Assign (Identifier,Exp)

Identifier(“x”) Call(Exp,Identifier,ExpList)

IdentifierExp(“y”) Identifier(“m”)

IntegerLiteral(1)

ExpList:[ , ]

Plus(Exp,Exp)

IntegerLiteral(4) (IntegerLiteral(5)

Cheng-Chia Chen 24

MiniJava : Grammar(I)

Program -> MainClass ClassDecl *

Program(MainClass, ClassDeclList) Program Goal() : { MainClass m; ClassDeclList cl = new ClassDeclList(); ClassDecl c; } { m = MainClass() ( c = ClassDecl() {cl.addElement(c);})* <EOF> {return new Program(m,cl) }

Cheng-Chia Chen 25

MiniJava : Grammar(II)

MainClass -> class id { public static void main ( String [] id )  

      { Statement } }

MainClass( Identifier, Identifier, Statement)

ClassDecl -> class id { VarDecl * MethodDecl * }

-> class id extends id { VarDecl* MethodDecl * }

ClassDeclSimple(…), ClassDecExtends(…)

VarDecl -> Type id ;

VarDecl(Type, Identifier)

MethodDecl -> public Type id ( FormalList )

       { VarDecl * Statement* return Exp ; } MethodDecl(Type,Identifier,FormalList,VarDeclList

StatementList, Exp)

Cheng-Chia Chen 26

MiniJava : Grammar(III)

FormalList -> Formal ( “,” Formal)*

Formal -> Type id Formal(Type, Identifier )

Type -> int [] IntArrayType()

->   boolean

->   int

->   id IdentifierType()

Cheng-Chia Chen 27

MiniJava : Grammar(IV)

Statement -> { Statement * }  

-> if ( Exp ) Statement else Statement

->  while ( Exp ) Statement  

-> System.out.println ( Exp ) ;  

-> id = Exp ;

->  id [ Exp ] = Exp ;

ExpList -> Exp (“,” Exp ) *

Cheng-Chia Chen 28

MiniJava : Grammar(V)

Exp -> Exp op Exp

->  Exp [ Exp ]  

-> Exp . length

->   Exp . Id ( ExpList )  

-> INTEGER_LITERAL 

-> true

-> false 

-> id 

-> this

-> new int [ Exp ]

-> new id ( )

->   ! Exp

->   ( Exp )