parser generation using slk and flex++ copyright © 2015 curt hill

Parser Generation

Using SLK and Flex++

Copyright © 2015 Curt Hill

Introduction• Previously we used SLK and Flex+

+ to generate a Scanner– SLK found the terminals– We used regular expressions and C

code to generate Flex++ input

• Now we have to finish the parser that uses it


Scanner Review 1• Here are the steps we went

through • Start with a BNF file

– Fed this into SLK

• Used the XXXConstants.h to identify terminal symbols

• Coded the definition section for Flex+– Directive, includes and useful regular

expressions


Scanner Review 2• For each terminal in the language we

made a Flex++ rule consisting of:– A regular expression– Some C++ code to process– Returned a token value

• Created a main program that executed the scanner– This could have been the third section

• Used make to execute Flex++ and then compile everything


Original Scanner Modified

• The original scanner took a file as a command line parameter

• Each token was displayed with the token itself and the token number

• Next we modify this to feed a parser– Our first task is to take all that display

stuff out– The scanner should just return values


SLK Again• When we fed the BNF into SLK it

generated a number of parser routines– 4 .cpp

• These need to be augmented with three classes that we need to write:– XXXToken– XXXError– XXXAction– Where XXX was supplied by the –n=

option of SLK


XXXToken Class• Next create the XXXToken class• This is the interface between the

parser and scanner• It has two public methods:

– int get() – int peek() or int peek()const;

• Both return a token but get consumes it and peek does not

• The peek function is used for the one token look ahead of LL(1) or LR(1) languages


Look Ahead


factor

constant ident ( )expression

When parsing a factor, we have three possible choices as to what comes next: A constant (which will be a numeric token) An identified (which is also a token) A left parenthesis (also a token)We use 1 token look ahed to determine what to do nextAn LR(3) or LL(3) language would require three token look ahead

XXXToken again

• This is the look ahead buffer• Parser calls get when it knows it

needs the next token• It calls peek when it is deciding

how to proceed– Peek returns the next token, but also

leaves it there for the next get


XXXError Class• Error recovery is sometimes a

sticky issue• Do you:

– Just quit there?– Attempt a recovery?– Generate an error message?

• The generalized parser cannot make these decisions, so leave it up to this class

• What methods are needed?Copyright © 2015 Curt Hill

XXXError Methods• int mismatch (int terminal, int token);– Called if token does not match what is

expected

• int no_entry (int nonterminal, int token, int level);– Called if the token is not among the

known tokens

• void input_left ();– Called if there is input after parse

thought it was complete

• void message(char *);– Called to display a message


Beyond Parsing

• One of the possibilities of generating a parser is a simple recognizer

• The recognizer scans the file and gives us a simple answer:– File was correctly formed or not

• In general we want more:– Semantic analysis– Semantic content


Semantic Analysis

• We want to know some things:– Was this variable defined before it

was used?– Was this variable of the correct type

for the expression it was in?

• These are things difficult or impossible to describe in BNF

• This is somewhere between syntax and semantics


Semantic Content• What does the program mean?

What is it telling us?• For a programming language we

must now either:– Generate equivalent code as a

compiler– Execute it as an interpreter

• In most SE applications we will do the latter

• This is why we need the action class


XXXAction Class• This class is used to communicate

that the parser has figured out something

• That something is one of the following:– It has entered a new state– It is about to do a reduction– It wants to execute an action

• Each of these corresponds to a method that you need to define for the class


State• The method is: void state(int state);

• The state tells us what the parser is currently working on

• It is the least useful of the methods for us


Reduction

• A reduction is when the parser has found everything on the RHS of a production and it is about to reduce it to the LHS of the production

• The signature for a reduce is:void reduce(int entry);

• The entry is a numbered production


Reduction Processing

• The first thing, from a debugging perspective, that needs to be done is display the production

• This is done with the following function:char * XXXGetProductionName(int entry);

• This provides a copy of the production


Augmented productions

• In the BNF that SLK accepts you may put actions to execute at the end of productions

• The format of the action is_action_X– The X is a number

• When the production is about to be reduced the execute function will get a call with this number


Example Augmented Production

• Here are two productions or one with alternation: return_type ::= void _action_7 type _action_8

• When the reduction void to type occurs you should be able to see the 7

• It is almost that easy– As we will see


Execute

• The signature is:void execute(int action_number);

• Every time one of the reductions is about to occur execute is called

• However, you always get a negative number and not the one you expect– If you add the value of the largest non-

terminal constant to the received value it equal the action number


Finally

• We generate a makefile to do everything but run it

• Next we do a demo• Then an assignment• Next lets consider the generated

parser next


parser generation using slk and flex++ copyright © 2015 curt hill

Documents