A High Performance Middleware in Java with a Real Application
Fabrice Huet*, Denis Caromel*, Henri Bal+
* Inria-I3S-CNRS, Sophia-Antipolis, France+ Vrije Universiteit, Amsterdam, Netherlands
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Motivations
• Offer an integrated solution for – Programming – Deploying– Running
applications in an heterogeneous environment
• Investigate the current situation of Java programming for high performance
Overview
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Jem3D• Solves 3D Maxwell equations
• Relies on finite volume approximation method
• Works on unstructured tetrahedral discretization of the computation domain
• In this paper, we compute the propagation of an eigenmode in a cubic metallic cavity (standard test case)
• The complexity of the computation is linked to the number of tetrahedra
• We set the number of points to be used for tetrahedra (Mesh size)
Jem3D (2)
Initialization
Compute mag. field, and update elec. field
Compute elec. field, and update mag. field
Calculation of the discreteelectromagnetic energy
Solution Saving
t < tmax
t = tmax
Communication intensive
Distribution of the ComputationExample on 4 nodes
N
1
4
1 1
2
2
Communication Area: 2*N2Communication Area: 3*N2
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
ProActive
• Java library for parallel, distributed and concurrent programming
• Medium grain entities: active object– Has its own thread of execution– Stores and serves requests– Asynchronous communications with transparent future objects
and wait by necessity
• Uses RMI or Soap-like protocol
• High level features: mobility, security, component, group communication, deployment descriptors…
Group Communications
• Idea / Goals– Manipulate groups as ‘simple’ Java objects– Maintain the ‘dot’ notation– Code reusability
• Remote method invocation on a group
• Multi-unicast approach– Based on the ProActive communication mechanism– Replication of n ‘single’ communication– Uses of multiple threads
Deployment Descriptor
• Abstract Away from source code:– Machines names– Creation/Connection Protocols– Lookup and Registry Protocols
• Interfaced with various protocols and Infrastructures:– Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols)– Internet and LAN: rsh, rlogin, ssh– Grid: Globus, Web services, gsissh
• API to query resources from within the application through symbolic names (VirtualNode)
Deployment Descriptor- ExamplesVirtualNode:
jem3dNode
Mapping:
jem3dNode VM1, VM2
Infrastructure:
VM1 Local VM
VM2 SSH host1
then Local VM
VirtualNode:
jem3dNode
Mapping:
jem3dNode VM1
Infrastructure:
VM1 SSH frontend
then pbsProcess
pbsProcess PBS 10 nodes Local VM
VM2VM1
localhost
host1
localhost frontend
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
App.
jem3dNode
App.
jem3dNode
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Ibis• Grid programming environment with efficient
communications
• Java Implementation for portability
• Native Implementation for performance– TCP– Myrinet– MPI
• Various programming models: Group Method Invocation (GMI), Divide and Conquer (Satin), Remote Method Invocation (RMI)
• We focus here on the Java implementation for portability reasons
Improved RMI implementation• Source compatible with Sun’s RMI (replace
java.rmi.* with ibis.rmi.*)
• Reduced overhead through caching of type information
• Overall, reduce by 10% the amount of data sent over the wire
• Improved serialization– Generation of serialization code for all classes– Specially generated constructor to create empty
objects
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Running Environment
• DAS-2 MultiCluster– 200 Nodes on 5 sites– ≥1GB of memory– We use 72 Nodes on 1 site, other evenly distributed– Intra domain nodes linked by 100Mb/s ethernet– Inter domains linked with 1Gb/s
• OS– RedHat 7.2
• Java– Sun’s 1.5.0 and IBM’s 1.4.1
Jem3D with 51x51x51 mesh
10
30
50
70
90
2 4 6 8 10 12 14 16 18 20Number of Nodes
Exe
cuti
on
Tim
e (s
)
ProActive/RMI IBM ProActive/RMI SunProActive/Ibis IBM ProActive/Ibis Sun
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Comparison with a Fortran Implementation
• Not a Java vs Fortran/MPI Comparison!• We compare:
– The same algorithm– Implemented in very different languages (dynamic
data structures in Java, static in Fortran)– By different persons– With different goals
• We want to know – How much we pay for the Java features– How much we can gain with Ibis
Running Environment• Cluster located in Sophia-Antipolis
– 16 nodes of Pentium 3 at 1Ghz– 512MB of RAM– Nodes linked with 100Mb/s Ethernet
• OS– Linux 2.4.17– Sun’s JDK 1.4.2
• Fortran/MPI– MPICH-1.2.5
Sequential Version
Time Memory Java/Fortran
Mesh Java Fortran Java Fortran Time Memory
21x21x21 45s 18.9s 78M 59M 2.38 1.32
31x31x31 150s 65s 224M 164M 2.30 1.36
41x41x41 387s 156s 483M 366M 2.48 1.31
Speedup, 51x51x51
1
3
5
7
9
11
13
2 4 6 8 10 12 14 16Number of Nodes
Sp
eed
up
Theorical Speedup Fortran/MPI 51x51x51ProActive/Ibis 51x51x51 ProActive/RMI 51x51x51
Speedup, 81x81x81
5
7
9
11
13
15
8 10 12 14 16Number of Nodes
Sp
eed
up
Theorical Speedup Fortran 81x81x81
ProActive/Ibis 81x81x81 ProActive/RMI 81x81x81
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
Conclusion
• ProActive + Ibis:– Integrated Solution– Allows a better speedup than plain RMI implementation– Gives to the application a better control on its environment
• Still slower (3x) than a “close” Fortran/MPI implementation but– Easier to compile– Easier to deploy (Jem3D ran on 150 nodes)
• Evaluate the native solutions– The native ibis version is installed/managed by an administrator– We compile against the Java classes only
Resources
• Jem3D: not available yet, but working on it
• ProActive: Available under LGPL– http://www-sop.inria.fr/oasis/ProActive
• Ibis: Available under LGPL– http://www.cs.vu.nl/ibis
• Booth #2345