grid computing @ sun€¦ · grid computing @ sun carlo nardone technical systems ambassador gso...
TRANSCRIPT
Grid Computing@ SunCarlo NardoneTechnical Systems AmbassadorGSO – Client Solutions
2cmn 2004-11
Phases of Grid Computing
Campus Grids• Multiple user communities• Single organization
Global Grids• Multiple user communities• Multiple organizations
Cluster Grids• Single user community• Single organization
3cmn 2004-11
Convergence of Grid and Web Services World● Sun is actively participating
– Platinum Sponsor of Global Grid Forum from its start– Enterprise Grid Alliance– OASIS
● Other standardization efforts– OGSA/OGSI
– WS – RF (Web Services Resource Framework)
– WS – GAF (Web Services Grid Application Framework)
– OMII (Open Middleware Infrastructure, UK)
– NMI (NSF Middleware Initiative)
4cmn 2004-11
Sun Grid Services Environment Small to Large Grid Computing Solutions
Sun QFS/SAM-FSSolaris ZFS, CacheFS
Solaris ContainersSolaris Resource Manager
Sun N1 Grid Engine 6
Cluster Grid Cluster Grid Infrastructure Infrastructure
Global Grid Global Grid Infrastructure Infrastructure
Enterprise Grid Enterprise Grid Infrastructure Infrastructure
Sun Management CenterSun Control Station
Service Service Discovery Discovery
Authentication/Authentication/Authorization Authorization
Data Data Management Management
Policy Policy Management Management
Resource Resource Management Management
System System Management Management
Data Data Access Access
Industry Standards and Industry Standards and partner technologies partner technologies
OGSA, WS-RF ... OGSA, WS-RF ... Globus ToolkitGlobus ToolkitAvaki, SRB ...Avaki, SRB ...
Sun N1 Provisioning System
Java Enterprise System, J2EEJava Enterprise System, J2EE
5cmn 2004-11
Java Enterprise System for EDU
6cmn 2004-11
JES Components
7cmn 2004-11
N1 Grid Engine Distributed Resource Management in Cluster & Enterprise Grids
● Multi–platform, open source– +10,000 downloads since 2000– Queueing, prioritizing, scheduling– Policy–based sharing btw groups & departments– Community project at gridengine.sunsource.net
● New features in N1GE 6– Improved throughput and scalability– New scheduler: resource reservation with backfilling– Standard DRMAA compliant– More supported platforms, Mac OS/X included– Optional BerkeleyDB–based logs– ARCo: Accounting & Reporting Console (Sun product only)– Future Windows exec client (Sun product only)
8cmn 2004-11
Grid Engine Portal An Open Source project based on Java Portal Server
9cmn 2004-11
Campus Grid Computing
10cmn 2004-11
A Successful Example● White Rose University Consortium
– Collaboration btw Univ. of York, Sheffield and Leeds– Critical mass for R&D amongst University departments
● White Rose Grid– In production since 2002– Globus Toolkit 2.0 middleware– GridEngine used for fair sharing of resources btw sites
GridProjects
12cmn 2004-11
Some Interesting Grid Project Using Sun Technologies● JGrid● JxGrid● GrIDE● Many others:
– EPCC TOG (Transfer-queue Over Globus), see gridengine.sunsource.net/project/gridengine/tog.html
– EPCC JOSH (JOb Scheduling Hierarchically) , see www.epcc.ed.ac.uk/sungrid/WP5/ and gridengine.sunsource.net/project/gridengine/josh.html
– JiniGrid, see http://pds.irt.vein.hu/jgrid/jgrid.html
13cmn 2004-11
JGrid
● Provides a useful remote API for submitting objects to the grid
● Lets Grid Engine do the scheduling and accounting
● Allows remote class loading● Minimizes overhead, enables scalability● Alpha available at – http://gridengine.sunsource.net/project/
gridengine/howto/JGrid/jgrid.html
A Java Interface to Grid Engine
14cmn 2004-11
JGrid Architecture
GEMaster
GE execdaemon
Gridlet
Gridlet
Native Peer
Native Peer
Com
pute
Pro
xyResultChannel
ResultChannel
Com
pute
Age
nt
Master Exec Host
Serialized Gridlet
Shared Storage
RMI interface
15cmn 2004-11
J2EE–JGrid Environment
App ServerN1GEMaster HostServlet Container
EJB Container
JGridAgent
execd
EJB
Servlet
Execution Host
EJBEJB
qmas
ter
JGri
d Se
rver
JGridAgent
execd
JGridAgent
execd
Execution Host
Execution Host
16cmn 2004-11
Project JxTA Peer–to–peer Networking via XML Protocols
17cmn 2004-11
JxTA Peer Groups
18cmn 2004-11
JxGrid A JXTA Approach to Resource Discovery for Grid Engine
19cmn 2004-11
GrIDE
● Developed at APSTC– Asia-Pacific Science & Technology Centers, a network
of R&D Centers in partnership btw Sun Microsystems EDU LOB and Academia
● GrIDE is an integrated development environment for Grids– Workflow Editor– Crosscompiler– Grid Debugger– Performance Tuning– Data Grid Access– Project Collaboration
20cmn 2004-11
GriDE Architecture
Middleware Layer
Programming languages/ Libraries Layer
GriDE Layer
NetBeans Tools Platform
Griddebugger
Data GridAccess
Java MPI-G2
SGE
ProjectCollaboration
Globus NimRod
Computing Resources
PerformanceTurning
Grid SimulationC/C++ Fortran
Flow Editor
AppsTemplates
Cross Compiler
Apps API
CoGKit
21cmn 2004-11
22cmn 2004-11
GrIDE Workflow Editor
DataGrids
24cmn 2004-11
Data Grids● Federated, transparent services, independent of
– Location– Name– Distribution– Replication– Ownership– Heterogeneity– Change of schema
● Support for domain–specific and general data access layer
● Replication management and consistency● Consistency services appropriate to nature of the Grid● Globus GridFTP is not enough!
25cmn 2004-11
Data Grids: EPCC OGSA–DAI● Main Grid effort at EPCC● Open Grid Services Architecture – Data Access
and Integration● Main tenet: while compute on demand is
important, it is a niche market● Distributed data on demand is much more
interesting prospect● ELDAS – J2EE version of OGSA–DAI● Check–out www.ogsadai.org.uk
26cmn 2004-11
Data Grids: Avaki
● Commercial result of Legion research project● Enterprise Information Integration● Unified data catalogue● Single data service layer● Access via Web services, ODBC, JDBC, file I/O● Caching for performance● Data integration accessible as a web service –
regardless of location● Available via Sun Client Solutions
27cmn 2004-11
Data Grids: SRB● Storage Resource Broker● Research project from SDSC, San Diego, see
www.npaci.edu/dice/srb
● Client–server middleware● Uniform interface for connecting to
heterogeneous data over networks to enable data grids, digital libraries, etc.
● Works with MCAT (Metadata Catalog) to:– Access data sets based on attributes and logical names– Various interfaces: inQ, MySRB, Jargon, Matrix
● Commercial version from Nirvana Storage
