1 simplifications of context-free grammars. 2 a substitution rule substitute equivalent grammar
TRANSCRIPT
![Page 1: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/1.jpg)
1
Simplifications of
Context-Free Grammars
![Page 2: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/2.jpg)
2
A Substitution Rule
bB
aAB
abBcA
aaAA
aBS
Substitute
Equivalentgrammar
aAB
abbcabBcA
aaAA
abaBS
|
|
bB
![Page 3: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/3.jpg)
3
A Substitution Rule
EquivalentgrammarabaAcabbcabBcA
aaAA
aaAabaBS
||
||
aAB
abbcabBcA
aaAA
abaBS
|
|
Substitute aAB
![Page 4: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/4.jpg)
4
In general:
1yB
xBzA
Substitute
zxyxBzA 1|equivalentgrammar
1yB
![Page 5: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/5.jpg)
5
Nullable Variables
:production A
Nullable Variable: A
![Page 6: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/6.jpg)
6
Removing Nullable Variables
Example Grammar:
M
aMbM
aMbS
Nullable variable
![Page 7: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/7.jpg)
7
M
M
aMbM
aMbSSubstitute
abM
aMbM
abS
aMbS
Final Grammar
![Page 8: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/8.jpg)
8
Unit-Productions
BAUnit Production:
(a single variable in both sides)
![Page 9: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/9.jpg)
9
Removing Unit Productions
Observation:
AA
Is removed immediately
![Page 10: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/10.jpg)
10
Example Grammar:
bbB
AB
BA
aA
aAS
![Page 11: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/11.jpg)
11
bbB
AB
BA
aA
aAS
SubstituteBA
bbB
BAB
aA
aBaAS
|
|
![Page 12: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/12.jpg)
12
Remove
bbB
BAB
aA
aBaAS
|
|
bbB
AB
aA
aBaAS
|
BB
![Page 13: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/13.jpg)
13
SubstituteAB
bbB
aA
aAaBaAS
||
bbB
AB
aA
aBaAS
|
![Page 14: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/14.jpg)
14
Remove repeated productions
bbB
aA
aBaAS
|
bbB
aA
aAaBaAS
||
Final grammar
![Page 15: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/15.jpg)
15
Useless Productions
aAA
AS
S
aSbS
aAaaaaAaAAS
Some derivations never terminate...
Useless Production
![Page 16: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/16.jpg)
16
bAB
A
aAA
AS
Another grammar:
Not reachable from S
Useless Production
![Page 17: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/17.jpg)
17
In general:
if wxAyS
then variable is usefulA
otherwise, variable is uselessA
)(GLw
contains only terminals
![Page 18: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/18.jpg)
18
A production is useless if any of its variables is useless
xA
DC
CB
aAA
AS
S
aSbS
Productions
useless
useless
useless
useless
Variables
useless
useless
useless
![Page 19: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/19.jpg)
19
Removing Useless Productions
Example Grammar:
aCbC
aaB
aA
CAaSS
||
![Page 20: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/20.jpg)
20
First: find all variables that can producestrings with only terminals
aCbC
aaB
aA
CAaSS
|| },{ BA
AS
},,{ SBA
Round 1:
Round 2:
![Page 21: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/21.jpg)
21
Keep only the variablesthat produce terminal symbols:
aCbC
aaB
aA
CAaSS
||
},,{ SBA
aaB
aA
AaSS
|
(the rest variables are useless)
Remove useless productions
![Page 22: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/22.jpg)
22
Second:Find all variablesreachable from
aaB
aA
AaSS
|
S A B
Use a Dependency Graph
notreachable
S
![Page 23: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/23.jpg)
23
Keep only the variablesreachable from S
aaB
aA
AaSS
|
aA
AaSS
|
Final Grammar
(the rest variables are useless)
Remove useless productions
![Page 24: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/24.jpg)
24
Removing All
Step 1: Remove Nullable Variables
Step 2: Remove Unit-Productions
Step 3: Remove Useless Variables
![Page 25: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/25.jpg)
25
Normal Formsfor
Context-free Grammars
![Page 26: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/26.jpg)
26
Chomsky Normal Form
Each productions has form:
BCA
variable variable
aAor
terminal
![Page 27: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/27.jpg)
27
Examples:
bA
SAA
aS
ASS
Not ChomskyNormal Form
aaA
SAA
AASS
ASS
Chomsky Normal Form
![Page 28: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/28.jpg)
28
Convertion to Chomsky Normal Form
Example:
AcB
aabA
ABaS
Not ChomskyNormal Form
![Page 29: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/29.jpg)
29
AcB
aabA
ABaS
Introduce variables for terminals:
cT
bT
aT
ATB
TTTA
ABTS
c
b
a
c
baa
a
cba TTT ,,
![Page 30: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/30.jpg)
30
Introduce intermediate variable:
cT
bT
aT
ATB
TTTA
ABTS
c
b
a
c
baa
a
cT
bT
aT
ATB
TTTA
BTV
AVS
c
b
a
c
baa
a
1
1
1V
![Page 31: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/31.jpg)
31
Introduce intermediate variable:
cT
bT
aT
ATB
TTV
VTA
BTV
AVS
c
b
a
c
ba
a
a
2
2
1
1
2V
cT
bT
aT
ATB
TTTA
BTV
AVS
c
b
a
c
baa
a
1
1
![Page 32: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/32.jpg)
32
Final grammar in Chomsky Normal Form:
cT
bT
aT
ATB
TTV
VTA
BTV
AVS
c
b
a
c
ba
a
a
2
2
1
1
AcB
aabA
ABaS
Initial grammar
![Page 33: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/33.jpg)
33
From any context-free grammar(which doesn’t produce )not in Chomsky Normal Form
we can obtain: An equivalent grammar in Chomsky Normal Form
In general:
![Page 34: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/34.jpg)
34
The Procedure
First remove:
Nullable variables
Unit productions
![Page 35: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/35.jpg)
35
Then, for every symbol : a
In productions: replace with a aT
Add production aTa
New variable: aT
![Page 36: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/36.jpg)
36
Replace any production nCCCA 21
with
nnn CCV
VCV
VCA
12
221
11
New intermediate variables: 221 ,,, nVVV
![Page 37: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/37.jpg)
37
Theorem:For any context-free grammar(which doesn’t produce )there is an equivalent grammar in Chomsky Normal Form
![Page 38: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/38.jpg)
38
Observations
• Chomsky normal forms are good for parsing and proving theorems
• It is very easy to find the Chomsky normal form for any context-free grammar
![Page 39: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/39.jpg)
39
Greinbach Normal Form
All productions have form:
kVVVaA 21
symbol variables
0k
![Page 40: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/40.jpg)
40
Observations
• Greinbach normal forms are very good for parsing
• It is hard to find the Greinbach normal form of any context-free grammar
![Page 41: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/41.jpg)
41
Compilers
![Page 42: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/42.jpg)
42
Compiler
Program
v = 5;if (v>5) x = 12 + v;while (x !=3) { x = x - 3; v = 10;}......
Add v,v,0cmp v,5jmplt ELSETHEN: add x, 12,vELSE:WHILE:cmp x,3...
Machine Code
![Page 43: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/43.jpg)
43
Lexicalanalyzer parser
Compiler
program machinecode
input output
![Page 44: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/44.jpg)
44
A parser knows the grammarof the programming language
![Page 45: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/45.jpg)
45
ParserPROGRAM STMT_LISTSTMT_LIST STMT; STMT_LIST | STMT;STMT EXPR | IF_STMT | WHILE_STMT | { STMT_LIST }
EXPR EXPR + EXPR | EXPR - EXPR | IDIF_STMT if (EXPR) then STMT | if (EXPR) then STMT else STMTWHILE_STMT while (EXPR) do STMT
![Page 46: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/46.jpg)
46
The parser finds the derivation of a particular input
10 + 2 * 5
Parser
E -> E + E | E * E | INT
E => E + E => E + E * E => 10 + E*E => 10 + 2 * E => 10 + 2 * 5
input
derivation
![Page 47: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/47.jpg)
47
10
E
2 5
E => E + E => E + E * E => 10 + E*E => 10 + 2 * E => 10 + 2 * 5
derivation
derivation tree
E E
E E
+
*
![Page 48: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/48.jpg)
48
10
E
2 5
derivation tree
E E
E E
+
*
mult a, 2, 5add b, 10, a
machine code
![Page 49: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/49.jpg)
49
Parsing
![Page 50: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/50.jpg)
50
grammar
Parserinputstring
derivation
![Page 51: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/51.jpg)
51
Example:
Parser
derivation
S
bSaS
aSbS
SSSinput
?aabb
![Page 52: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/52.jpg)
52
Exhaustive Search
||| bSaaSbSSS
Phase 1:
S
bSaS
aSbS
SSSaabb
All possible derivations of length 1
Find derivation of
![Page 53: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/53.jpg)
53
S
bSaS
aSbS
SSS aabb
![Page 54: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/54.jpg)
54
Phase 2
aSbS
SSS
aabb
SSSS
bSaSSSS
aSbSSSS
SSSSSS
Phase 1
abaSbS
abSabaSbS
aaSbbaSbS
aSSbaSbS
||| bSaaSbSSS
![Page 55: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/55.jpg)
55
Phase 2
SSSS
aSbSSSS
SSSSSS
aaSbbaSbS
aSSbaSbS
Phase 3
aabbaaSbbaSbS
||| bSaaSbSSS
aabb
![Page 56: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/56.jpg)
56
Final result of exhaustive search
Parser
derivation
S
bSaS
aSbS
SSSinput
aabb
aabbaaSbbaSbS
(top-down parsing)
![Page 57: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/57.jpg)
57
Time complexity of exhaustive search
Suppose there are no productions of the form
A
BA
Number of phases for string : approx. |w| w
![Page 58: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/58.jpg)
58
Time for phase 1: k
k possible derivations
For grammar with rules k
![Page 59: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/59.jpg)
59
Time for phase 2: 2k
possible derivations2k
![Page 60: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/60.jpg)
60
Time for phase |w| is 2|w|:
A total of 2|w| possible derivations
![Page 61: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/61.jpg)
61
Total time needed for string :w
||2 wkkk
Extremely bad!!!
phase 1 phase 2 phase |w|
![Page 62: 1 Simplifications of Context-Free Grammars. 2 A Substitution Rule Substitute Equivalent grammar](https://reader033.vdocuments.us/reader033/viewer/2022061305/5513f6cd55034646298b6327/html5/thumbnails/62.jpg)
62
For general context-free grammars:
There exists a parsing algorithmthat parses a stringin time
||w3||w
The CYK parser