a high performance middleware in java with a real application
DESCRIPTION
A High Performance Middleware in Java with a Real Application. Fabrice Huet *, Denis Caromel*, Henri Bal + * Inria-I3S-CNRS, Sophia-Antipolis, France + Vrije Universiteit, Amsterdam, Netherlands. Outline. Motivations and Overview Jem3D: 3D electromagnetism - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/1.jpg)
A High Performance Middleware in Java with a Real Application
Fabrice Huet*, Denis Caromel*, Henri Bal+
* Inria-I3S-CNRS, Sophia-Antipolis, France+ Vrije Universiteit, Amsterdam, Netherlands
![Page 2: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/2.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 3: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/3.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 4: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/4.jpg)
Motivations
• Offer an integrated solution for – Programming – Deploying– Running
applications in an heterogeneous environment
• Investigate the current situation of Java programming for high performance
![Page 5: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/5.jpg)
Overview
![Page 6: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/6.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 7: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/7.jpg)
Jem3D• Solves 3D Maxwell equations
• Relies on finite volume approximation method
• Works on unstructured tetrahedral discretization of the computation domain
• In this paper, we compute the propagation of an eigenmode in a cubic metallic cavity (standard test case)
• The complexity of the computation is linked to the number of tetrahedra
• We set the number of points to be used for tetrahedra (Mesh size)
![Page 8: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/8.jpg)
Jem3D (2)
Initialization
Compute mag. field, and update elec. field
Compute elec. field, and update mag. field
Calculation of the discreteelectromagnetic energy
Solution Saving
t < tmax
t = tmax
Communication intensive
![Page 9: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/9.jpg)
Distribution of the ComputationExample on 4 nodes
N
1
4
1 1
2
2
Communication Area: 2*N2Communication Area: 3*N2
![Page 10: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/10.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 11: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/11.jpg)
ProActive
• Java library for parallel, distributed and concurrent programming
• Medium grain entities: active object– Has its own thread of execution– Stores and serves requests– Asynchronous communications with transparent future objects
and wait by necessity
• Uses RMI or Soap-like protocol
• High level features: mobility, security, component, group communication, deployment descriptors…
![Page 12: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/12.jpg)
Group Communications
• Idea / Goals– Manipulate groups as ‘simple’ Java objects– Maintain the ‘dot’ notation– Code reusability
• Remote method invocation on a group
• Multi-unicast approach– Based on the ProActive communication mechanism– Replication of n ‘single’ communication– Uses of multiple threads
![Page 13: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/13.jpg)
Deployment Descriptor
• Abstract Away from source code:– Machines names– Creation/Connection Protocols– Lookup and Registry Protocols
• Interfaced with various protocols and Infrastructures:– Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols)– Internet and LAN: rsh, rlogin, ssh– Grid: Globus, Web services, gsissh
• API to query resources from within the application through symbolic names (VirtualNode)
![Page 14: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/14.jpg)
Deployment Descriptor- ExamplesVirtualNode:
jem3dNode
Mapping:
jem3dNode VM1, VM2
Infrastructure:
VM1 Local VM
VM2 SSH host1
then Local VM
VirtualNode:
jem3dNode
Mapping:
jem3dNode VM1
Infrastructure:
VM1 SSH frontend
then pbsProcess
pbsProcess PBS 10 nodes Local VM
VM2VM1
localhost
host1
localhost frontend
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
App.
jem3dNode
App.
jem3dNode
![Page 15: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/15.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 16: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/16.jpg)
Ibis• Grid programming environment with efficient
communications
• Java Implementation for portability
• Native Implementation for performance– TCP– Myrinet– MPI
• Various programming models: Group Method Invocation (GMI), Divide and Conquer (Satin), Remote Method Invocation (RMI)
• We focus here on the Java implementation for portability reasons
![Page 17: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/17.jpg)
Improved RMI implementation• Source compatible with Sun’s RMI (replace
java.rmi.* with ibis.rmi.*)
• Reduced overhead through caching of type information
• Overall, reduce by 10% the amount of data sent over the wire
• Improved serialization– Generation of serialization code for all classes– Specially generated constructor to create empty
objects
![Page 18: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/18.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 19: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/19.jpg)
Running Environment
• DAS-2 MultiCluster– 200 Nodes on 5 sites– ≥1GB of memory– We use 72 Nodes on 1 site, other evenly distributed– Intra domain nodes linked by 100Mb/s ethernet– Inter domains linked with 1Gb/s
• OS– RedHat 7.2
• Java– Sun’s 1.5.0 and IBM’s 1.4.1
![Page 20: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/20.jpg)
Jem3D with 51x51x51 mesh
10
30
50
70
90
2 4 6 8 10 12 14 16 18 20Number of Nodes
Exe
cuti
on
Tim
e (s
)
ProActive/RMI IBM ProActive/RMI SunProActive/Ibis IBM ProActive/Ibis Sun
![Page 21: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/21.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 22: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/22.jpg)
Comparison with a Fortran Implementation
• Not a Java vs Fortran/MPI Comparison!• We compare:
– The same algorithm– Implemented in very different languages (dynamic
data structures in Java, static in Fortran)– By different persons– With different goals
• We want to know – How much we pay for the Java features– How much we can gain with Ibis
![Page 23: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/23.jpg)
Running Environment• Cluster located in Sophia-Antipolis
– 16 nodes of Pentium 3 at 1Ghz– 512MB of RAM– Nodes linked with 100Mb/s Ethernet
• OS– Linux 2.4.17– Sun’s JDK 1.4.2
• Fortran/MPI– MPICH-1.2.5
![Page 24: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/24.jpg)
Sequential Version
Time Memory Java/Fortran
Mesh Java Fortran Java Fortran Time Memory
21x21x21 45s 18.9s 78M 59M 2.38 1.32
31x31x31 150s 65s 224M 164M 2.30 1.36
41x41x41 387s 156s 483M 366M 2.48 1.31
![Page 25: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/25.jpg)
Speedup, 51x51x51
1
3
5
7
9
11
13
2 4 6 8 10 12 14 16Number of Nodes
Sp
eed
up
Theorical Speedup Fortran/MPI 51x51x51ProActive/Ibis 51x51x51 ProActive/RMI 51x51x51
![Page 26: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/26.jpg)
Speedup, 81x81x81
5
7
9
11
13
15
8 10 12 14 16Number of Nodes
Sp
eed
up
Theorical Speedup Fortran 81x81x81
ProActive/Ibis 81x81x81 ProActive/RMI 81x81x81
![Page 27: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/27.jpg)
Outline• Motivations and Overview
• Jem3D: 3D electromagnetism
• ProActive: High Level Middleware for the Grid
• Ibis: High Performance Communications
• Java experimentations
• Comparison with a Fortran Implementation
• Conclusion
![Page 28: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/28.jpg)
Conclusion
• ProActive + Ibis:– Integrated Solution– Allows a better speedup than plain RMI implementation– Gives to the application a better control on its environment
• Still slower (3x) than a “close” Fortran/MPI implementation but– Easier to compile– Easier to deploy (Jem3D ran on 150 nodes)
• Evaluate the native solutions– The native ibis version is installed/managed by an administrator– We compile against the Java classes only
![Page 29: A High Performance Middleware in Java with a Real Application](https://reader035.vdocuments.us/reader035/viewer/2022070400/568134e8550346895d9c2190/html5/thumbnails/29.jpg)
Resources
• Jem3D: not available yet, but working on it
• ProActive: Available under LGPL– http://www-sop.inria.fr/oasis/ProActive
• Ibis: Available under LGPL– http://www.cs.vu.nl/ibis
• Booth #2345