parsingparsing. 2 front-end: parser checks the stream of words and their parts of speech for...
TRANSCRIPT
ParsingParsingParsingParsing
2
Front-End: ParserFront-End: ParserFront-End: ParserFront-End: Parser
Checks the stream of words and their parts of speech for grammatical correctness
scanner parsersourcecode
tokens IR
errors
3
Front-End: ParserFront-End: ParserFront-End: ParserFront-End: Parser
Determines if the input is syntactically well formed
scanner parsersourcecode
tokens IR
errors
4
Front-End: ParserFront-End: ParserFront-End: ParserFront-End: Parser
Guides context-sensitive (“semantic”) analysis (type checking)
scanner parsersourcecode
tokens IR
errors
5
Front-End: ParserFront-End: ParserFront-End: ParserFront-End: Parser
Builds IR for source program
scanner parsersourcecode
tokens IR
errors
6
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysis Natural language analogy:
consider the sentence
He wrote the program
7
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysis
He wrote the program
noun verb article noun
8
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysis
He wrote the program
noun verb article noun
subject predicate object
9
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysis Natural language analogy
He wrote the program
noun verb article noun
subject predicate object
sentence
10
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysis Programming language
if ( b <= 0 ) a = b
bool expr assignment
if-statement
11
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysissyntax errors
int* foo(int i, int j)){ for(k=0; i j; ) fi( i > j ) return j;}
Compiler Compiler ConstructionConstruction
Compiler Compiler ConstructionConstruction
Sohail Aslam
Lecture 11
13
Syntactic AnalysisSyntactic AnalysisSyntactic AnalysisSyntactic Analysisint* foo(int i, int j))
{
for(k=0; i j; )
fi( i > j )
return j;
}
extra parenthesis
Missing expression
not a keyword
14
Semantic AnalysisSemantic AnalysisSemantic AnalysisSemantic Analysis Grammatically correct
He wrote the computer
noun verb article noun
subject predicate object
sentence
15
Semantic AnalysisSemantic AnalysisSemantic AnalysisSemantic Analysis semantically (meaning) wrong!
He wrote the computer
noun verb article noun
subject predicate object
sentence
16
Semantic AnalysisSemantic AnalysisSemantic AnalysisSemantic Analysisint* foo(int i, int j){ for(k=0; i < j; j++ ) if( i < j-2 ) sum = sum+i return sum;}
undeclared var
return type
mismatch
17
Role of the ParserRole of the ParserRole of the ParserRole of the Parser Not all sequences of tokens
are program. Parser must distinguish
between valid and invalid sequences of tokens.
18
Role of the ParserRole of the ParserRole of the ParserRole of the Parser Not all sequences of tokens
are program. Parser must distinguish
between valid and invalid sequences of tokens.
19
Role of the ParserRole of the ParserRole of the ParserRole of the ParserWhat we need
An expressive way to describe the syntax
An acceptor mechanism that determines if input token stream satisfies the syntax
20
Role of the ParserRole of the ParserRole of the ParserRole of the ParserWhat we need
An expressive way to describe the syntax
An acceptor mechanism that determines if input token stream satisfies the syntax
21
Role of the ParserRole of the ParserRole of the ParserRole of the ParserWhat we need
An expressive way to describe the syntax
An acceptor mechanism that determines if input token stream satisfies the syntax
22
Study of ParsingStudy of ParsingStudy of ParsingStudy of Parsing Parsing is the process of
discovering a derivation for some sentence
23
Study of ParsingStudy of ParsingStudy of ParsingStudy of Parsing Mathematical model of
syntax – a grammar G.
Algortihm for testing membership in L(G).
24
Study of ParsingStudy of ParsingStudy of ParsingStudy of Parsing Mathematical model of
syntax – a grammar G.
Algortihm for testing membership in L(G).
25
Context Free GrammarsContext Free GrammarsContext Free GrammarsContext Free GrammarsA CFG is a four tuple
G=(S,N,T,P) S is the start symbol N is a set of non-terminals T is a set of terminals P is a set of productions
26
Why Not Regular Why Not Regular Expressions?Expressions?Why Not Regular Why Not Regular Expressions?Expressions?Reason:
regular languages do not have enough power to express syntax of programming languages.
27
Limitations of Regular Limitations of Regular LanguagesLanguagesLimitations of Regular Limitations of Regular LanguagesLanguages
Finite automaton can’t remember number of times it has visited a particular state
28
Example of CFGExample of CFGExample of CFGExample of CFG
Context-free syntax is specified with a CFG
29
Example of CFGExample of CFGExample of CFGExample of CFG Example
SheepNoise → SheepNoise baa| baa
This CFG defines the set of noises sheep make
30
Example of CFGExample of CFGExample of CFGExample of CFG We can use the
SheepNoise grammar to create sentences
We use the productions as rewriting rules
31
Example of CFGExample of CFGExample of CFGExample of CFGSheepNoise → SheepNoise baa
| baa
Rule Sentential Form- SheepNoise2 baa
32
Example of CFGExample of CFGExample of CFGExample of CFGSheepNoise → SheepNoise baa
| baa
Rule Sentential Form- SheepNoise1 SheepNoise baa2 baa baa
33
Example of CFGExample of CFGExample of CFGExample of CFG
And so on ...
Rule Sentential Form- SheepNoise1 SheepNoise baa1 SheepNoise baa baa2 baa baa baa
34
Example of CFGExample of CFGExample of CFGExample of CFG While it is cute, this
example quickly runs out intellectual steam
To explore uses of CFGs, we need a more complex grammar
35
Example of CFGExample of CFGExample of CFGExample of CFG While it is cute, this
example quickly runs out intellectual steam
To explore uses of CFGs, we need a more complex grammar
36
More Useful GrammarMore Useful GrammarMore Useful GrammarMore Useful Grammar1 expr → expr op expr2 | num3 | id4 op → +5 | –6 | *7 | /
37
Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF)
Grammar rules in a similar form were first used in the description of the Algol60 Language.
38
Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF) The notation was developed
by John Backus and adapted by Peter Naur for the Algol60 report.
Thus the term Backus-Naur Form (BNF)
39
Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF)Backus-Naur Form (BNF) The notation was developed
by John Backus and adapted by Peter Naur for the Algol60 report.
Thus the term Backus-Naur Form (BNF)
40
Derivation:Derivation:Derivation:Derivation: Let us use the expression
grammar to derive the sentence
x – 2 * y
41
Derivation: Derivation: x – 2 x – 2 ** y yDerivation: Derivation: x – 2 x – 2 ** y yRule Sentential Form
- expr1 expr op expr2 <id,x> op expr5 <id,x> – expr1 <id,x> – expr op
expr
42
Derivation: Derivation: x – 2 x – 2 ** y yDerivation: Derivation: x – 2 x – 2 ** y y
Rule Sentential Form2 <id,x> – <num,2> op
expr6 <id,x> – <num,2>
expr3 <id,x> – <num,2>
<id,y>
43
DerivationDerivationDerivationDerivation Such a process of rewrites
is called a derivation.
Process or discovering a derivations is called parsing
44
DerivationDerivationDerivationDerivation Such a process of rewrites
is called a derivation.
Process or discovering a derivations is called parsing
45
DerivationDerivationDerivationDerivation
We denote this derivation as:
expr →* id – num * id
46
DerivationsDerivationsDerivationsDerivations At each step, we choose a
non-terminal to replace
Different choices can lead to different derivations.
47
DerivationsDerivationsDerivationsDerivations At each step, we choose a
non-terminal to replace
Different choices can lead to different derivations.
48
DerivationsDerivationsDerivationsDerivations Two derivations are of
interest
1. Leftmost derivation
2. Rightmost derivation
49
DerivationsDerivationsDerivationsDerivations Leftmost derivation:
replace leftmost non-terminal (NT) at each step
Rightmost derivation: replace rightmost NT at each step
50
DerivationsDerivationsDerivationsDerivations Leftmost derivation:
replace leftmost non-terminal (NT) at each step
Rightmost derivation: replace rightmost NT at each step
51
DerivationsDerivationsDerivationsDerivations The example on the
preceding slides was leftmost derivation
There is also a rightmost derivation
52
Rightmost DerivationRightmost DerivationRightmost DerivationRightmost DerivationRule Sentential Form
- expr1 expr op expr3 expr op <id,x>6 expr <id,x>1 expr op expr
<id,x>
53
Derivation: Derivation: x – 2 x – 2 ** y yDerivation: Derivation: x – 2 x – 2 ** y y
Rule Sentential Form2 expr op <num,2>
<id,x>5 expr – <num,2>
<id,x>3 <id,x> – <num,2>
<id,y>
54
DerivationsDerivationsDerivationsDerivations In both cases we have
expr →* id – num id
55
DerivationsDerivationsDerivationsDerivations The two derivations produce
different parse trees.
The parse trees imply different evaluation orders!
56
DerivationsDerivationsDerivationsDerivations The two derivations produce
different parse trees.
The parse trees imply different evaluation orders!