reducing interprocess communication overhead in concurrent programs erik stenman kostis sagonas
TRANSCRIPT
Reducing Interprocess Communication Overhead
in Concurrent Programs
Erik StenmanKostis Sagonas
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 2
Motivation Concurrency as an abstraction is
important. Systems that need to interact with the outside
world are hard to model without concurrency. Unfortunately concurrency costs.
There are two types of runtime overheads: ”Direct overhead” of concurrency primitives. ”Indirect overhead” from hiding the data-flow from
the optimizing compiler.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 3
Goal of this work Reduce the overhead of concurrency
in concurrent programs.
Idea Optimize the code that implements
process communication.We call this interprocess optimization, and we will present three techniques:1. Rescheduling send.2. Direct dispatch send3. Interprocess inlining.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 4
Rescheduling Send Typically, when a process sends a
message, it is because it wants the receiver to act on that message.
It is therefore in the interest of the sender to yield to the receiver and allow it to promptly act on the sent message.
We call this type of send, a rescheduling send.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 5
Implementation: The send operation suspends the
currently active (sending) process.
Benefits: lower message passing latency. better cache behavior (the receiver has
access to the message while it is still hot in the cache).
Rescheduling Send
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 6
Direct Dispatch Send The sender contributes the remaining part of its
time-slice to the receiver (hoping this would lead to a faster response).
The receiver then is woken up directly (ignoring the ready queue). Overhead of the scheduler is eliminated.
If the receiver also performs a direct dispatch send back to the sender, interprocess communication becomes as fast as a function call.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 7
Interprocess Inlining Merge code from the sender with
code from the receiver.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 8
Process The receiver
Process The sender
A message
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 9
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 10
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 11
Known communication protocol.Can be optimized.Process only needs to be ready to receive the communication.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 12
The state of process has changed.
Without really participating in the actual communication.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 13
Interprocess Inlining Merge code from the sender with
code from the receiver. In the merged code, the overhead of
the communication protocol can be optimized away.
We suggest using profiling to find situations where this is feasible.
This requires that the optimized code is guarded by a runtime test.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 14
An example in Erlang…ref_server(V) -> receive {‘++’,From}-> From !
{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !
{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !
{self(),V}, ref_server(V) end
inc(Beta) ->
Beta ! {‘++’,self()},
receive
{Beta,Answer} ->
Answer;
end.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 15
…could be rewritten asref_server(V) -> receive {‘++’,From}-> From !
{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !
{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !
{self(),V}, ref_server(V) end
inc´(Beta) ->
if ready(Beta)->
B_V=get(‘V’,Beta)+1,
save(‘V’,B_V,Beta),
B_V;
true ->
inc(Beta)
end.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 16
Code merging: the general case
g - Head
Createmessage
Send
-Tail
g - Head
Receive
g – tail
’ - Head
Createmessage
Send
-Tail
Extractedg – tail
Ready
Copy
Restoreβ-State
Saveβ-State
Copy of-Tail
Yes
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 17
Code merging
Can only be done if: Code explosion does not occur. The code does not suspend. The control flow does not escape the
included code. The extracted code terminates
(o/w, the sending process might hang after code merging.)
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 18
Code merging These instructions are not extracted:
A call or a return. Instructions that lead to the suspension of the
process. Throwing an exception. Some BIFs are large and uncommon and not
worth the effort to adapt for external execution.
Non-terminating code is unacceptable. Hence, we do not accept any loops in the
control flow graph.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 19
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
Createmessage
tail
send
f(Beta) ->
Beta ! {hi,self()},
receive
{Beta,fine} ->
right;
{Beta,_Other} ->
wrong
end.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 20
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
Test
f-tail
f(Beta) -> Mess = {hi,self()}, if ready(Beta)-> NEWCODE; true -> Beta ! Mess, receive {Beta,fine} -> right; {Beta,_Answer} -> wrong endend.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 21
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
case -Mbox of [{hi, -From}]-> -From ! -{-self(),fine};
_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 22
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
case [{hi,self()}]of [{hi, -From}]-> -From ! -{-self(),fine};
_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 23
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
-From = self(), -From ! -{-self(),fine},
save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 24
Code merging
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
-From = self(), self()! -{-self(),fine},
save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 25
Return messages An important special case Handled by applying the same type
of rewriting. The ready test is replaced by a test
that checks that both mailboxes are empty.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 26
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
-From = self(), self()! -{-self(),fine},
save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 27
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = [{hi,self()}],
-From = self(), -Mbox = {-self(),fine},
save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 28
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE:-Mbox = {-self(),fine},
save_state(Beta),COPY-OF-f-TAIL
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 29
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = {-self(),fine},
save_state(Beta),
case -Mbox of {Beta,fine} ->
right;
{Beta,_Other} ->
wrong
end
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 30
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = {-self(),fine},
save_state(Beta),case {-self(),fine} of
{Beta,fine} -> right; {Beta,_Other} -> wrong end
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 31
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: -Mbox = {-self(),fine},
save_state(Beta),
right
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 32
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
NEWCODE: save_state(Beta),
right
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 33
More Code mergingg() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
f´(Beta) -> if ready(Beta)->
save_state(Beta), right; true -> f(Beta)end.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 34
This optimization gives Access to variables in another
process with almost no overhead. (Two reads, one test, and two writes.)
The overhead of the communication protocol can be reduced. (Creating a
tuple and switching on it.)
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 35
Profiling We have profiled parts of Eddie and
the AXD code and found: That each send goes to one particular
receive. The receiving process is typically
suspended with an empty mailbox. The same profiling tool could be used in a
dynamic compiler to find pairs of senders/receivers whose inter-process communication can be optimized.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 36
Conclusions Presented several different methods
for cross-process optimization that reduce the overhead of interprocess communication.
No modifications to existing code are required.
Open issue is their integration into the Erlang development environment
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 37
Praeterea censeo "0xCA" scribere Erlang posse.Happi
That’s all folks!
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 38
Problem How does one find senders and
receivers that communicate? Static analysis
Problematic in a language with dynamic typing, dynamic process creation, communication through mailboxes, and asynchronous communication.
Profiling & dynamic recompilation
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 39
Interprocess inlining Do:
Find two communicating processes. Merge code from the sender with code
from the receiver. Optimize the merged code.
Get: Reduced message passing. Short-circuited code for switching on
messages.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 40
An example in Erlang…
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
f(Beta) ->
Beta ! {hi,self()},
receive
{Beta,fine} ->
right;
{Beta,_Other} ->
wrong
end.
Info
rmati
onst
ekn
olo
gi
ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 41
…could be rewritten as
g() ->
receive
{hi,From}->
From ! {self(),fine};
_ -> ignore
end,
g().
f´(Beta) -> if ready(Beta)->
save_state(Beta), right; true -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end end.