Software Overhead in Messaging Layers

Pitch Patarasuk

Messaging Layer

• Bridge between the hardware functionality and the user communication requirement

• Network hardware feature– Arbitrary delivery order– Finite buffering– Limited fault handling

• User communication requirement– In-order delivery– End-to-end flow control– Reliable transmission

Messaging Layer (cont.)

Software overhead

• Main communication cost = hardware cost + software cost

• Software cost dominates hardware cost

• 50-70% of the software cost are the direct gap between the network features and user requirements

Software overhead in messaging layers

• Analyze the costs of communication functionality and network features

• What overhead might be reduced if the underlining network provide higher level of service

CM-5 active message layer (CMAM)• CM-5

– Parallel machine up to a few thousand nodes connected with an incomplete fat tree topology

• Active message

– Communication mechanism intended to expose full hardware flexibility and performance of modern networks

– Each message contain an address of a user-level handler which is executed on message arrival with the message body as an argument

– The handler extract data from the network and integrate it into the ongoing computation

• CMAM vs Send/Recv– User direct access to network, no OS involve, the data go directly to the

user space computation

Software overhead cost analysis

• Consider implementation of 3 protocols– Single-packet delivery– Finite sequence, multi-packet delivery– Indefinite sequence, multi-packet delivery

• Use instruction counts for measurement

Single-packet delivery

Description Source Destination

Call/Return 3 10

NI Setup - -

Write to NI 2 -

Read from NI - 3

Check NI status 7 12

Control Flow 3 2

20 27

Finite sequence, multi-packet delivery

Indefinite sequence, multi-packet delivery

Message size = 16 words

Message size = 1024 words

Messaging layer with high-level network feature

• Given that CMAM is considered to be very efficient, there are 2 choices to reduce software overhead– Lower user requirement– Raise level of service provided by the network

• Compressionless Routing (CR)– Order-preserving transmission– Deadlock freedom independent of packet acceptance

guarantees– Fault-tolerant transmission at packet level

Single-packet delivery

• Has the same cost as the previous CMAM case

• However, it is guaranteed to be fault-free, no deadlock or buffer overflow

Finite sequence, multi-packet delivery

Indefinite sequence, multi-packet delivery

Message size = 16 words

Message size = 1024 words

Discussion

• Larger packet sizes– Reduce overhead, especially for indefinite-sequence

protocol

• Improved network interfaces and DMA hardware– Network interface:

• only make basic cost faster, but not protocol cost in messaging layer

• Make it more important for messaging layer to be effective

– DMA:• only reduce cost in moving large amounts of data

Discussion (Cont.)

• Implication of network design– Improving routing performance may increase software

cost, e.g. out-of-order delivery

• Reducing communication features– Fault-tolerance, in-order-delivery

– Put burden on parallel software programmer

• Use instruction counts for measurement instead of latency– Latency is hard to measure in a portable fashion

Conclusion

• Software overhead is much larger than routing time and 50-70% are messaging layer overhead

• Software overhead in messaging layer is the cost of the gap between the network feature and the user requirement

• This cost can be reduced to zero if the underlining network provides functionality that the user requires.

Questions?