hack your language!...since resume behaves like a call, do we need to introduce a separate construct...
TRANSCRIPT
![Page 1: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/1.jpg)
1
Lecture 5: Bytecode Compiler
Implementing CoroutinesCompile AST to bytecodeBytecode interpreter
Ras Bodik Alvin CheungMaaz Ahmad
Talia RingerBen Tebbs
Hack Your Language!CSE401 Winter 2016Introduction to Compiler Construction
![Page 2: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/2.jpg)
Announcements
• HW2 and PA2 will be out next Monday
• Maaz will have OH after class
2
![Page 3: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/3.jpg)
What you will learn today
3
parserprogram PL desugarASTL AST-based interpreterASTC
output from P
input to P
parserprogram PL desugarASTL
bytecode interpreter
ASTC
output from P
input to P
compiler bytecodeC
AST Interpreter
Bytecode compiler + Bytecode Interpreter
![Page 4: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/4.jpg)
What you will learn today
Implement Asymmetric Coroutines– why a recursive interpreter with implicit stack won’t do
Compiling AST to bytecode- this is your first compiler; compiles AST to “flat” code
Bytecode Interpreter- bytecode can be interpreted without recursion- hence no need to keep interpreter state on the call stack
4
parsercode textL desugarASTL AST-based interpreterASTC
output from P
input to P
![Page 5: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/5.jpg)
Taking a Closer Look at Coroutines
5
![Page 6: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/6.jpg)
Review: Why are coroutines useful?
Loop calls the iterator function to get the next iteme.g., the next token from the input stream
The iterator maintains state between two such callse.g., pointer to the input stream
Maintenance of that state may be difficultsee the permutation iterator in previous lecture
Industrial-strength justification of Python generatorsrestricted Python’s coroutines
6
![Page 7: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/7.jpg)
Review: Three coroutine constructs
co = coroutine(body) function → handlecreates a coroutine whose body is the argument function,returns a handle to the coroutine, which is ready to run
yv = resume(co, rv) handle × value →valueresumes the execution into the coroutine co, passing rv to co’s yield expression (rv becomes the value of yield)
rv = yield(yv) value → valuetransfers control to the master (ie, the coroutine that resumed into the current coroutines), passing yv to resume
7
![Page 8: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/8.jpg)
Example
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
8
def g(x,y) {ck=coroutine(k)def h(a,b) {
3 + yield(resume(ck,a,b))}h(x,y)
}
def k(x,y) {yield(x+y)
}
The call f(1,2) evaluates to __.
![Page 9: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/9.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 10: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/10.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 11: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/11.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 12: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/12.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 13: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/13.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 14: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/14.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 15: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/15.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 16: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/16.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 17: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/17.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}
![Page 18: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/18.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}3
![Page 19: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/19.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}3
![Page 20: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/20.jpg)
What was going on
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) { 3 + yield( resume(ck,a,b))
}
h(x,y) }
def k(x,y) {
yield(x+y)
}3
3
![Page 21: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/21.jpg)
A corner-case contest
Identify cases for which we haven’t defined behavior:
1) yield executed in the main programlet’s define main as a coroutine; yield at the top level thus behaves as exit()
2) resuming to itself*
illegal; can only resume to coroutines suspended in yield expressions or at the beginning of their body
3) return statementthe (implicit) return statement yields back to resumer and terminates the coroutine
*exercise: test your PA2 on such a program 21
![Page 22: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/22.jpg)
States of a coroutine
How many states do we want to distinguish? suspended, running, terminated
Why do we want to distinguish them?to perform error checking at runtime:
- do not resume into a running coroutine- do not resume into a terminated coroutine
22
![Page 23: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/23.jpg)
Are coroutines like calls?
Which of { resume, yield } behaves like a function call?
resume is more like call:- like in regular call, control is guaranteed to return to
resume (unless the program runs into an infinite loop)- we can specify the target of the call (via coro. handle)
yield is more like return:- like return, yield cannot control where it returns (always
returns to its resumer’s resume expression)- no guarantee that the coroutine will be resumed after
yield (eg, when a loop decides it need not iterate further)
23
![Page 24: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/24.jpg)
Example from last lecture
Find the best first move in Scrabble given some time:
s = [‘a‘,‘f‘,...] // 7 letter tilesfor p in permgen(s) {
for s in subsets(p) {if (legalWord(s, wordDict)) {
// check score of s // exit when out if time
}}
}
24
![Page 25: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/25.jpg)
Test yourself
Implement lazy list concatenation(http://bit.ly/1U0Eaqv)
25
![Page 26: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/26.jpg)
Language design question (1)
Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this?
co = coroutine(…) // creates a coroutineco(arg) // resume to the coroutine
Yes, we could. But we will keep resume for clarity.Compare: in Python generators, resume is a method call .next() in a generator object. Do you like the fact that resume appears to be a call?
26
![Page 27: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/27.jpg)
Language design question (2)
In 164/Lua, a coroutine is created with corutine(lam).In Python, a coroutine is created by a call to function that syntactically contains a yield:
def fib(): # function fib is a generator/coroua, b = 0, 1while 1:
yield b # … because it contains a yielda, b = b, a+b
it = fib() # create a coroutineprint it.next(); # resume to it with .next()print it.next(); print it.next()
27
![Page 28: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/28.jpg)
Language design question (2, cont’d)
Python creates coroutine by calling a function with yield? How does it impact programming?
What if the function with yield needs to call itself recursively, as in permgen from Lecture 4?
def permgen(a,n) { ... yield(a) ... permgen(a,n-1) ... }def permutations(a) {
def co = coroutine( permgen )lambda () { resume(co, a) }
}
You should know a workaround from this limitationThat is, how would you write permgen in Python?See how Python writes recursive tree generators
28
![Page 29: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/29.jpg)
Exercise on Lua vs. Python
A recursive Lua Fibonacci iterator:
def fib(a,b) {yield bfib(b,a+b)
}def fibIterator() {
def co = coroutine( fib )lambda () { resume(co, 0, 1) }
}
Rewrite it into a recursive Python generator.29
![Page 30: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/30.jpg)
Symmetric vs. asymmetric coroutines
Symmetric: one construct (yield)– as opposed to resume and yield– yield(co,v): has the power to transfer to any coroutine– all transfers happen with yield (no need for resume)
Asymmetric: – resume transfers from resumer (master) to corou (slave)– yield(v) always returns to its resumer
30
![Page 31: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/31.jpg)
A language sufficient to explain coroutines
Language with functions of two arguments:
P ::= D*, E sequence of declarations followed by an expression
D ::= def f(ID,ID) { P }
E ::= n | ID(E,E) | def ID = coroutine(ID) | resume(ID,E,E) | yield(E)
31
![Page 32: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/32.jpg)
Implementation notes
Stackless Python is a controversial rethinking of the Python core (PEP 255)
32
![Page 33: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/33.jpg)
Summary
Coroutines support backtracking. In match, backtracking happens in seq, where we pick a match for patt1 and try to extend it with a match for patt2. If we fail, we backtrack and pick the next match for patt1, and again see if it can be extended with a match for patt2.
33
![Page 34: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/34.jpg)
Test yourself
Draw the state of our coroutine interpreter (link to an excercise)
34
![Page 35: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/35.jpg)
Why a recursive interpreter cannot implement (deep) coroutines
35
![Page 36: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/36.jpg)
The plan
We will attempt to extend Section 2 to support coroutines
treating resume/yield as call/return
This extension will failmotivating a non-recursive interpreter architecture
This failure should be unsurprisingbut will clarify the difference between PA2 and Sec 2 interpreters
36
![Page 37: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/37.jpg)
Example
37
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
![Page 38: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/38.jpg)
Example
38
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fn
![Page 39: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/39.jpg)
Example
39
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fn
t1
![Page 40: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/40.jpg)
Example
40
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fn
t1t2
![Page 41: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/41.jpg)
Example
41
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fn
t1t2
![Page 42: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/42.jpg)
Example
42
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1v2
p
![Page 43: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/43.jpg)
Example
43
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1v2
fsym valueparent
p
![Page 44: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/44.jpg)
Example
44
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1v2
sym valueparent
x 1
p
f
![Page 45: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/45.jpg)
Example
45
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1v2
sym valueparent
x 1y 2
f
p
![Page 46: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/46.jpg)
Example
46
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(fn.env, fn, t1, t2)} def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1v2
sym valueparent
x 1y 2
f
p
![Page 47: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/47.jpg)
The recursive Sec 2 interpreter
Program is represented as an AST– evaluated by walking the AST bottom-up
State (context) of the interpreter:- control: what AST nodes to evaluate next (the “PC”)- data: intermediate values (arguments to operators, calls)
Where is this state maintained?- control: on interpreter’s calls stack (one PC per level)- data: in interpreter’s variables (eg, t1, t2)
47
![Page 48: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/48.jpg)
Example
48
call
add a
fun arg1
b
arg2
add
+
id id
body
arg1 arg2
“x” “y”name
sym valueadda 1b 2
AST env
function add(x,y) { x + y }a = 1b = 2add(a, b)
case ‘call’: // E(E,E)def fn = eval(node.fun)def t1 = eval(node.arg1)def t2 = eval(node.arg2)
return call(env, fn, t1, t2)}def call(p, fun, v1, v2) { f = Frame(p)f.push(fun.body.arg1, v1)f.push(fun.body.arg2, v2)eval(fun.body, f)}
fun
v1 v2
sym valueparent
x 1y 2
f
pInterpreter state
Call stack
![Page 49: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/49.jpg)
An attempt for a recursive coro. interpreter
def eval(node) {switch (node.op) {case ‘resume’: // resume E, E
evaluate the expression that gives coroutine handle -- OK
def co = eval(n.arg1)def t1 = eval(n.arg2)now resume to c/r by evaluating its body(resumes in the right point only on first resume to co)
call(co.body, t1)case ‘yield’: // yield E
return eval(node.arg) oops: this returns to eval() in the same coroutine, rather than to the resumer coroutine as was desired
} } 49
![Page 50: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/50.jpg)
Draw call stacks at this point
50
call(envf, f, 1, 2)eval(env1, resume)resume(env2, cg, 1, 2)
call(envg, g, 1, 2)call(envh, h, 1, 2)eval(env2, add)eval(env2, yield)resume(env2, ck, 1, 2)
call(envk, k, 1, 2)eval(env3, yield)
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) {
3 + yield(resume(ck,a,b))}h(x,y)
}
def k(x,y) {yield(x+y)
}
![Page 51: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/51.jpg)
Draw call stacks at this point
51
Draw the calls stacks in the interpreter:
The interpreter's call stack contains the recursive eval() calls.- the control context (what code to execute next)
The interpreter's environment has variables t1 and t2 (see prev slide).- the data context (values of intermediate values)
def f(x,y) {cg=coroutine(g)resume(cg,x,y)
}
f(1,2)
def g(x,y) {ck=coroutine(k)def h(a,b) {
3 + yield(resume(ck,a,b))}h(x,y)
}
def k(x,y) {yield(x+y)
}
![Page 52: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/52.jpg)
Draw call stacks at this point
52
call(envf, f, 1, 2)eval(env1, resume)resume(env2, cg, 1, 2)
call(envg, g, 1, 2)call(envh, h, 1, 2)eval(env2, add)eval(env2, yield)resume(env2, ck, 1, 2)
call(envk, k, 1, 2)eval(env3, yield)
resume(env2, cg, 1, 2) resume(env2, ck, 1, 2)
envfenv1env2
envgenvhenv2
envkenv3
recursive interpreter (can’t yield without losing state)
bytecode interpreter (keeps state on the stack of each coroutine)
![Page 53: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/53.jpg)
Summary
If it exits when yielding, it loses this contextthis exit would need to pop all the way from recursion
The interpreter for a coroutine c must return– when c encounters a yield
The interpreter will be called again– when some coroutine invokes resume(c, …)
53
![Page 54: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/54.jpg)
Summary
A recursive, Sec 2-like interpreter cannot yieldWe need a separate call stack for each coroutine.To preserve the coroutine context while it is suspended.
Why multiple call stacks are impossible in PA2 interp?The interpreter can only use one call stack – because the host language (JS) does not have threads or coroutines.
54
![Page 55: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/55.jpg)
Intermission
55
![Page 56: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/56.jpg)
The real deal
56
![Page 57: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/57.jpg)
Must re-architect the interpreter
We need to create a non-recursive interpreter- non-recursive ==> yield can become a return to resumer
all the context stored in separate data structures- explicit stack data structure is OK
57
![Page 58: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/58.jpg)
The plan
Linearize the AST so that the control context can be kept as a program counter (PC)
a stack of these PCs will be used to implement call/return
Store intermediate values in program’s variablesin PA2, they were in the interpreter’s variables
These two goals are achieved by translating the AST into register-based bytecode
58
![Page 59: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/59.jpg)
The plan
59
AST-based interpreter
output from P
input to P
parserprogram PL desugarASTL
bytecode interpreter
ASTC
output from P
input to P
compiler bytecodeC
![Page 60: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/60.jpg)
AST-to-bytecode compiler
60
![Page 61: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/61.jpg)
AST vs. bytecode
Abstract Syntax Tree:values of sub-expressions do not have explicit names
Bytecode: – list of instructions of the form x = y + z– x, y, z can be temporary variables invented by compiler– temporaries store values that would be in the variables
of a recursive interpreter
Example: AST (x+z)*y translates to bytecode:
$1 = x+z // $1, $2: temp vars$2 = $1*y // return value of AST is in $2
61
![Page 62: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/62.jpg)
Example
62
+
x z
arg1 arg2
*
y
arg1 arg2
$1 = x+z // $1, $2: temp vars$2 = $1*y // return value of AST is in $2
(x + z) * y
$1 = x+z// $1: temp var
![Page 63: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/63.jpg)
Compile AST to bytecode
Traverse AST a, emitting code c rather than evaluatingwhen the generated code c is later executed, it evaluates the original AST a
We’ll write a translation function b with signature:b : AST -> (Code, RegName)
The function produces code c and register name r s.t.:when c is executed, the value of AST is stored in register r
63
![Page 64: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/64.jpg)
Compile AST to bytecode (cont)def b(n):
switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s = %s + %s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s = %s\n” % (r, n.val), r)
ID: variable name (allows compiling x+3+y)return (“”, n.name)
=: ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” % (n.left.name, r),
n.left.name)
64
![Page 65: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/65.jpg)
Example
65
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=right left
z
variable value
rightleft
rightleft
![Page 66: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/66.jpg)
Example
66
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value
right left
rightleft
rightleft
![Page 67: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/67.jpg)
Example
67
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value
right left
rightleft
rightleft
![Page 68: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/68.jpg)
Example
68
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable valuereturn value (“”, x)
right left
rightleft
rightleft
![Page 69: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/69.jpg)
Example
69
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) (“”, x)
right left
rightleft
rightleft
![Page 70: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/70.jpg)
Example
70
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable valuer $1
right left
rightleft
rightleft
![Page 71: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/71.jpg)
Example
71
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable valuer $1
return value ($1=10, $1)
right left
rightleft
rightleft
![Page 72: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/72.jpg)
Example
72
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) (“”, x)
(c2, r2) ($1=10, $1)
r $2
right left
rightleft
rightleft
![Page 73: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/73.jpg)
Example
73
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) (“”, x)
(c2, r2) ($1=10, $1)
r $2
instr $2=x+$1
right left
rightleft
rightleft
![Page 74: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/74.jpg)
Example
74
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) (“”, x)
(c2, r2) ($1=10, $1)
r $2
instr $2=x+$1
return value ( $1=10$2=x+$1, $2 )
right left
rightleft
rightleft
![Page 75: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/75.jpg)
Example
75
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) ( $1=10
$2=x+$1, $2 )
right left
rightleft
rightleft
![Page 76: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/76.jpg)
Example
76
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable valuereturn value (“”, y)
right left
rightleft
rightleft
![Page 77: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/77.jpg)
Example
77
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) ( $1=10
$2=x+$1, $2 )
(c2, r2) (“”, y)
r $3
right left
rightleft
rightleft
![Page 78: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/78.jpg)
Example
78
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) ( $1=10
$2=x+$1, $2 )
(c2, r2) (“”, y)
r $3
instr $3=$2+y
right left
rightleft
rightleft
![Page 79: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/79.jpg)
Example
79
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c1, r1) ( $1=10
$2=x+$1, $2 )
(c2, r2) (“”, y)
r $3
instr $3=$2+y
return value $1=10( $2=x+$1$3=$2+y , $3)
right left
rightleft
rightleft
![Page 80: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/80.jpg)
Example
80
z = (x + 10) + y
def b(n): switch n.op: +: (c1,r1) = b(n.left)
(c2,r2) = b(n.right)def r = freshReg()def instr = “%s=%s+%s\n” % (r, r1, r2)return (c1 + c2 + instr, r)
n: // integer literal (allows compiling 2+3+4)def r = freshReg()return (“%s=%s\n” % (r, n.val), r)
ID: // variable name (allows compiling x+3+y)return (“”, n.name)
=: // ID = E (allows compiling x=1+3+x)(c,r) = b(n.right)return (c + “%s=%s\n” %
(n.left.name, r), n.left.name)
+
x 10
+
y
=
z
variable value(c, r) $1=10
( $2=x+$1$3=$2+y , $3)
return value $1=10( $2=x+$1 , z )$3=$2+yz=$3
right left
rightleft
rightleft
![Page 81: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/81.jpg)
Summary (1)
Bytecode compiler: two tasks- translates AST into flat code (aka straight-line code)- benefit: interpreter does not need to remember which
part of the tree to evaluate next
Our bytecode is a “register-based bytecode”- it stores intermediate values in registers- these are virtual, not machine registers- we implement them as local variables
There is also stack-based bytecode- intermediate values pushed and popped onto data stack
81
![Page 82: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/82.jpg)
Summary (2)
Bytecode translation recursively linearizes the AST- the sub-translation picks a fresh temp variable and
returns its name along with the generated code - alternatively, caller tells the sub-translations in which
temp variable the generated code must store its result
Translation to machine code not very different- Sometimes done several nodes at a time, to find an
instruction that executes both nodes
82
![Page 83: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/83.jpg)
Bytecode interpreter
83
![Page 84: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/84.jpg)
Bytecode interpreter
Data (ie, environments for lexical scoping): same as in PA2
Control: coroutine: each c/r is a separate interpreter
- these (resumable) interpreters share data environments- each has own control context (call stack + next PC)
more on control in two slides …
first, let’s examine the typical b/c interpreter loopit’s a loop now, not a recursive call
84
![Page 85: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/85.jpg)
The bytecode interpreter loop
Note: this loop does not yet support resume/yield
def eval(bytecode_array) { // program to be executedpc = 0while (true) {
curr_instr = bytecode_array[pc]switch (curr_instr.op) {case ‘+’: … // do + on args (as in PA2)
pc++ // jump to next bc instr…}
}}
85
![Page 86: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/86.jpg)
Control, continued
call: push PC to call stackset PC to the start of the target function
return:pops the PC from call stacksets PC to it
86
![Page 87: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/87.jpg)
Control, continued
yield v: return from interpreterpassing v to resume
- PC after yield remains stored in c/r control context
resume c, v: restart the interpreter of cpass v to c
87
![Page 88: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/88.jpg)
Discussion
Not quite a non-recursive interpreterwhile calls are handled with the interpreter loop, each resume recursively calls an interpreter instance
What do we need to store in the c/r handle?- a pointer to an interpreter instance, which stores - the call stack- the next PC
88
![Page 89: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/89.jpg)
Summary (1)
Control context:- index into a bytecode array
Call / return: do not recursively create a sub-interpeter- call: add the return address to the call stack- return: jump to return address popped from call stack
89
![Page 90: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/90.jpg)
Summary (2)
Create a coroutine: create a sub-interpreter- initialize its control context and environment - so that first resume start evaluating the coroutine- the coroutine is suspended, waiting for resume
Resume / yield- resume: restarts the suspended coroutine- yield: store the context, go to suspend mode, return
yield value
90
![Page 91: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/91.jpg)
Test yourself
Extend the bytecode interpreter on slide 50 so that it supports yield and resume. Hint: Note that this interpreter always starts at pc=0. Change eval so that it can start at any pc.
91
![Page 92: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/92.jpg)
Reading
Required:see the reading for Lecture 4
Recommended:Implementation of Python generators
Fun:continuations via continuation passing style
92
![Page 93: Hack Your Language!...Since resume behaves like a call, do we need to introduce a separate construct to resume into a coroutine? Could we just do this? co = coroutine(…) // creates](https://reader033.vdocuments.us/reader033/viewer/2022042109/5e8a0065a1699c0d11233ba7/html5/thumbnails/93.jpg)
Summary
Implement Asymmetric Coroutines– why a recursive interpreter with implicit stack won’t do
Compiling AST to bytecode- btw, compilation to assembly is pretty much the same
Bytecode Interpreter- can exit without losing its state
93