lr(k) grammar
DESCRIPTION
LR(k) Grammar. David Rodriguez-Velazquez CS6800-Summer I, 2009 Dr. Elise De Doncker. Bottom-Up Parsing. Start at the leaves and grow toward root As input is consumed, encode possibilities in an internal state A powerful parsing technology LR grammars - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/1.jpg)
LR(k) Grammar
David Rodriguez-VelazquezCS6800-Summer I, 2009
Dr. Elise De Doncker
![Page 2: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/2.jpg)
Bottom-Up Parsing
• Start at the leaves and grow toward root• As input is consumed, encode possibilities in
an internal state• A powerful parsing technology• LR grammars– Construct right-most derivation of program– Left-recursive grammar, virtually all programming
language are left-recursive– Easier to express syntax
![Page 3: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/3.jpg)
Bottom-Up Parsing
• Right-most derivation– Start with the tokens– End with the start symbol– Match substring on RHS of production, replace by
LHS– Shift-reduce parsers• Parsers for LR grammars• Automatic parser generators (yacc, bison)
![Page 4: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/4.jpg)
Bottom-Up Parsing• Example Bottom-Up Parsing
S S + E | EE num | (S)
(1+2+(3+4))+5 (E+2+(3+4))+5 (S+2+(3+4))+5
(S+E+(3+4))+5 (S+(3+4))+5 (S+(E+4))+5
(S+(S+4))+5 (S+(S+E))+5 (S+(S))+5
(S+E)+5 (S)+5 E+5
S+5 S+E S
![Page 5: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/5.jpg)
Bottom-Up Parsing
• Advantage– Can postpone the selection of productions until
more of the input is scanned
Top-DownParsing
Bottom-UpParsing
SS + E | EE num | (S)
More time to decide what rules to apply
![Page 6: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/6.jpg)
Terminology LR(k)
• Left-to-right scan of input• Right-most derivation • k symbol lookahead• [Bottom-up or shift-reduce] parsing or LR
parser• Perform post-order traversal of parse tree
![Page 7: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/7.jpg)
Shift-Reduce Parsing
• Parsing actions: – A sequence of shift and reduce operations
• Parser state:– A stack of terminals and non-terminals (grows to
the right)
• Current derivation step:= stack + input
![Page 8: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/8.jpg)
Shift-Reduce Parsing
Derivation Step(Stack + input)
Stack(terminals &
non-terminals)
Unconsumed input
(1+2+(3+4))+5 (1+2+(3+4))+5 shift
(E+2+(3+4))+5 (E +2+(3+4))+5 reduce
(S+2+(3+4))+5 (S +2+(3+4))+5 reduce
(S+E+(3+4))+5 (S+E +(3+4))+5 reduce
![Page 9: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/9.jpg)
Shift-Reduce Actions• Parsing is a sequence of shift and reduces• Shift: move look-ahead token to stack
• Reduce: Replace symbols from top of stack with non-terminal symbols X corresponding to the production: X β (e.g., pop β, push X)
Stack Input Action
( 1+2+(3+4))+5 Shift 1
(1 +2+(3+4))+5
Stack Input Action
(S+E +(3+4)+5 Reduce SS+E
(S +(3+4)+5
![Page 10: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/10.jpg)
Shift-Reduce ParsingDerivation Stack Input stream Action
(1+2+(3+4))+5 (1+2+(3+4))+5 shift
(1+2+(3+4))+5 ( 1+2+(3+4))+5 shift
(1+2+(3+4))+5 (1 +2+(3+4))+5 reduce E num
(E+2+(3+4))+5 (E +2+(3+4))+5 reduce S E
(S+2+(3+4))+5 (S +2+(3+4))+5 Shift
(S+2+(3+4))+5 (S+ 2+(3+4))+5 Shift
(S+2+(3+4))+5 (S+2 +(3+4))+5 reduce E num
(S+E+(3+4))+5 (S+E +(3+4))+5 reduce S S + E
(S+(3+4))+5 (S +(3+4))+5 Shift
(S+(3+4))+5 (S+ (3+4))+5 Shift
(S+(3+4))+5 (S+( 3+4))+5 Shift
(S+(3+4))+5…
(S+(3 +4))+5 reduce E num
SS + E | EE num | (S)
![Page 11: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/11.jpg)
Potential Problems
• How do we know which action to take: whether to shift or reduce, and which production to apply
• Issues– Sometimes can reduce but should not– Sometimes can reduce in different ways
![Page 12: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/12.jpg)
Action Selection Problem
• Given stack β and loock-ahead symbol b, should parser:– Shift b onto the stack making it βb?– Reduce X γ assuming that the stack has the form β
= αγ making it αX ?
• If stack has the form αγ, should apply reduction Xγ (or shift) depending on stack prefix α ?– α is different for different possible reductions since γ’s
have different lengths
![Page 13: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/13.jpg)
LR Parsing Engine
• Basic mechanism– Use a set of parser states– Use stack with alternating symbols and states– Use parsing table to:• Determine what action to apply (shift/reduce)• Determine next state
– The parser actions can be precisely determined from the table
![Page 14: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/14.jpg)
LR Parsing Table
Action Table Goto Table
State Terminals Non-Terminals
State number Next action and next state Next State
![Page 15: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/15.jpg)
LR Parsing Table Example S(L) | idL S | L,S
![Page 16: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/16.jpg)
LR (k) Grammars
• LR(k) = Left-to-right scanning, right most derivation, k lookahead chars
• Main cases– LR(0) and LR(1)– Some variations SLR and LALR(1)
• Parsers for LR(0) Grammars:– Determine the action without any lookahead– Will help us understand shift-reduce parsing
![Page 17: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/17.jpg)
Building LR(0) Parsing Table• To build the parsing table– Define states of the parser– Build a DFA to describe transitions between states– Use the DFA to build the parsing table
• Each LR(0) state is a set of LR(0) items– An LR(0) item: X α.β where X αβ is a production
in the grammar– The LR(0) items keep track of the progress on all of
the possible upcoming productions– The item X α.β abstracts the fact that the parser
already matched the string α at the top of the stack
![Page 18: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/18.jpg)
Example LR(0) State
• An LR(0) item is a production from the language with a separator “.” somewhere in the RHS of the production
• Sub-string before “.” is already on the stack (beginnings of possible γ‘s to be reduced)
• Sub-string after “.”: what we might see next
Enum.E(.S)
stateitem
![Page 19: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/19.jpg)
Start State and Closure
• Start state– Augment grammar with production: S’ S$– Start state of DFA has empty stack: S’ .S$
• Closure of a parser state:– Start with Closure(S) =S– Then for each item in S:• X α.γβ• Add items for all the productions Y γ to the closure
of S: Y . γ
![Page 20: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/20.jpg)
Closure Example
• Set of possible production to be reduced next• Added items have the “.” located at the
beginning no symbols for these items on the stack yet
S(L) | idL S | L,S
DFA start state
S’ . S $
S’ . S $S . ( L )S . id
closure
![Page 21: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/21.jpg)
The Goto Operation• Goto operation : describes transitions
between parser states, which are sets of items• Algorithm: for state S and a symbol Y – If the item [X α . Y β] is in I(state), then – Goto(I,Y) = Closure([X α . Y β] )
S’ . S $S . ( L )S . id
Goto(S,’(‘) Closure( { S ( . L ) } )
![Page 22: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/22.jpg)
Shift: Terminal Symbols
In new state, include all items that have appropriate input symbol just after dot, advance dot in those items and take closure
S’ . S $S . ( L )S . id
GrammarS(L) | idL S | L,S S ( . L )
L . SL . L , SS . ( L )S . id
S id .
(
idid (
![Page 23: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/23.jpg)
Goto: Non-terminal Symbols
Same algorithm for transitions on non-terminals
S’ . S $S . ( L )S . id
GrammarS(L) | idL S | L,S
S ( . L )L . SL . L , SS . ( L )S . id
S id .
(
idid (
L S .
S
LS ( L . )L L . , S
![Page 24: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/24.jpg)
Applying Reduce Actions
Pop RHS off stack, replace with LHS X (X β ), then rerun DFA
S’ . S $S . ( L )S . id
GrammarS(L) | idL S | L,S
S ( . L )L . SL . L , SS . ( L )S . id
S id .
(
idid (
L S .
S
LS ( L . )L L . , S
States causing reduction(dot has reached the end)
![Page 25: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/25.jpg)
Full DFA
S’ . S $S . ( L )S . id
GrammarS(L) | idL S | L,S
S ( . L )L . SL . L , SS . ( L )S . id
S id .
(
idid
)
LS ( L . )L L . , S
S’ S . $
Final state
S
$
L S .
S
L L , . SS . ( L )S . id
,
id
S ( L ) . (
S L , S .
S2
13
5
47
6
8
9(
![Page 26: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/26.jpg)
LR Parsing Table Example S(L) | idL S | L,S
![Page 27: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/27.jpg)
Building the Parsing Table
• States in the table = states in the DFA• For transitions S S’ on terminal C:– Table [S,C] = Shift(S’) where S’ = next state
• For transitions S S’ on non-terminal N– Table [S,N] = Goto(S’)
• If S is a reduction state X β then:– Table [S, *] = Reduce(x β)
![Page 28: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/28.jpg)
Parsing Algorithm
• Algorithm: look at entry for current state s and input terminal a– If Table[s, a] = shift then shift:• Push(t), let ‘a’ be the next input symbol
– If Table[s, a] = Xα then reduce:• Pop(| α |) , t = top(), push(GOTO[t,X]), output X α
![Page 29: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/29.jpg)
Reductions• On reducing X β with stack α β– Pop β off stack, revealing prefix α and state– Take single step in DFA from top state– Push X onto stack with new DFA state
• Example:
Derivation Stack Input Action
((a),b 1(3(3 a),b) shift, goto 2
((a),b 1(3(3 a 2 ),b) reduce S id
((S),b a(3(3 S 7 ),b) reduce L S
![Page 30: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/30.jpg)
LR(0) Summary
• LR(0) parsing recipe:– Start with LR(0) grammar– Compute LR(0) states and build DFA• Use the closure operation to compute states• Use the goto operation to compute transitions
– Build the LR(0) parsing table from the DFA
• This can be done automatically– Parser Generator Yacc
![Page 31: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/31.jpg)
Question: Full DFA for this grammar
S’ . S $S . ( L )S . id
GrammarS(L) | idL S | L,S
S ( . L )L . SL . L , SS . ( L )S . id
S id .
(
idid
)
LS ( L . )L L . , S
S’ S . $
Final state
S
$
L S .
S
L L , . SS . ( L )S . id
,
id
S ( L ) . (
S L , S .
S2
13
5
47
6
8
9
(
![Page 32: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/32.jpg)
References
• Aho A.V., Ullman J. D., Lam M., Sethi R., “Compilers Principles, Techniques & Tools”, Second Edition,Addison Wesley
• Sudkamp A. T., An Introduction the Theory of Computer Science L&M, third edition, Addison Wesley
![Page 33: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/33.jpg)
![Page 34: LR(k) Grammar](https://reader036.vdocuments.us/reader036/viewer/2022062323/56815b17550346895dc8c7da/html5/thumbnails/34.jpg)
Question: Parsing ((a),b)Derivation Stack Input stream Action((a),b) 1 ((a),b)$ shift, goto 3
((a),b) 1(3 (a),b)$ shift, goto 3
((a),b) 1(3(3 a),b)$ shift, goto 2
((a),b) 1(3(3a2 ),b)$ reduce S id
((S),b) 1(3(3S7 ),b)$ reduce L S
((L),b) 1(3(3L5 ),b)$ shift, goto 6
((L),b) 1(3(3L5)6 ,b)$ reduce S (L)
(S,b) 1(3S7 ,b)$ reduce L S
(L,b) 1(3L5 ,b)$ shift, goto 8
(L,b) 1(3L5,8 b)$ shift, goto 9
(L,b) 1(3L5,8b2 )$ reduce S id
(L,S) 1(3L5,8S9 )$ reduce L L,S
(L) 1(3L5 )$ shift, goto 6
(L) 1(3L5)6 $ reduce S (L)
S 1S4 $ Done
S(L) | idL S | L,S