lishep grid workshop - february 2002 - uerj, rio de janeiro 1 regional center and grid development...
TRANSCRIPT
![Page 1: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/1.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
1
Regional Center and Grid Development in Brazil
Cláudio [email protected]
GPPD - Parallel and Distributed Computing Group
Instituto de Informática - UFRGS
Brazil
![Page 2: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/2.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
2
Authors• Alberto Santoro• phd and master students of GPPD
– Adenauer Yamin Corrêa - UCPel and UFRGS
– Luciano Cavalheiro da Silva - UFRGS
– Patrícia Kayser Vargas - La Salle, UFRJ and UFRGS
– Marko Petek - UFRGS
– Diana Adamatti - UFRGS
– Iara Augustin - UFSM and UFRGS
– Jorge Barbosa - UCPel and UFRGS
![Page 3: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/3.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
3
Contents
• Brazilian Grid group• HEP: Grid for high energy physics• GPPD projects
– DOAP: Grid middleware• adaptive scheduling
– PRIMOS: high performance Java
– VIC ++: adaptable VIC for Access Grid
• Other Brazilian Grid Projects• Final remarks
![Page 4: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/4.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
4
Brazilian Grid Group
• First national meeting– organized by CNPq– October 2001– Finep– members of CENAPADs– application users (HEP, LNCC, ...)– researchers on high-performance computing– RNP: research national network
![Page 5: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/5.jpg)
5
Brazilian Grid Group
• Goals– promote the use of Grid
• 3 groups– hw infrastructure
• network, systems (clusters), ...
– sw organization• middleware, accounting, security, ...
– applications
![Page 6: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/6.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
6
Brazilian Grid Group
• Some initial propositions (documents)– first meeting report– proposition for sw organization– proposition for applications– some discussions on important points
• A Grid email list• http://www.inf.ufrgs.br/procpar/hetnos/grid/
papers/
![Page 7: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/7.jpg)
7
Brazilian Network• Brazil - USA
– 155 Mbps– Ampath (Miami), Internet 2: 45 Mbps– no link to Europe
• RS - SP/RJ: 45 Mbps
• UERJ - CBPF: 8 Mbps
• CBPF - RNP: 100 Mbps (local)
• SP - RJ: 10 Mbs
• RS: 1 - 2 Mbps
![Page 8: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/8.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
8
Brazilian Network
• Submitted project to Funttel– Gigabit network - GbE/WDM– Unicamp, CPqD/RNP, USP, INPE, LNCC,
UFRJ, PUC-RJ, UFF– 2002-2003
![Page 9: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/9.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
9
High Energy Physics - HEP
Brazilian Grid
Coordination - Alberto Santoro
![Page 10: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/10.jpg)
![Page 11: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/11.jpg)
![Page 12: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/12.jpg)
![Page 13: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/13.jpg)
13
HEP
• Current state– UERJ (A. Santoro)
• a grant from Faperj• February 2002• a cluster with (perhaps) 32 nodes
– UFRGS• has two small clusters
– Myrinet, fastethernet, SCI• bulding a new 32 nodes Myrinet cluster• mainly for development and testing
![Page 14: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/14.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
14
DOAP
• Grid-like middleware
• Distributed object oriented programming
• Adaptive scheduling
• Mobile computing
![Page 15: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/15.jpg)
15
DOAP Concerns
System Heterogeneity
GRID Computing
CLUSTER Computing
System's Components Availability
Mobility (Hardware and
Software)
Adaptation (Aplication and
System)
![Page 16: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/16.jpg)
16
DOAP: a pervasive view
A pervasive view in
GRID Computing
Mobile Computing
Wireless
GRID Computing
Wide-area
Meta-computing
High performance equips.
Adaptability
High heterogeneity
![Page 17: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/17.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
17
GRID Computing in DOAP
Development Decisions
Execution Decisions
Adaptation Centric Scheduling
Applications
Scheduling(Adaptation
Management)
ResourceMetrics
Execution Environment Components (processors, databases,process, services, etc)
Status information(capacity, utilization, etc.)
ResourcesDemand
AdaptationAlternatives
AdaptationControl
AdaptationControl
ResourceAvailabilityand Demand
![Page 18: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/18.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
18
DOAP Architecture
Holoparadigm
Multiparadigm;Blackboards;Mobility.
Distributed Mobile Application
ISAMadapt (Holoparadigm)
ADAPTATION
User VirtualEnvironment SCHEDULING
Execution Profile Resource Profile User Profile
NamingComuni-cation
Migra-tion
Repli-cation
Interope-rability
Loca-tion
Monito-ring
Java Virtual Machine
Native Operational System
Static Network + Mobile Network
![Page 19: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/19.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
19
Mobility: Hardware and Software
Language level adaptation constructors
DOAP Architecture
Distributed Mobile Application
ISAMadapt (Holoparadigm)
ADAPTATION
User VirtualEnvironment SCHEDULING
Execution Profile Resource Profile User Profile
NamingComuni-cation
Migra-tion
Repli-cation
Interope-rability
Loca-tion
Monito-ring
Java Virtual Machine
Native Operational System
Static Network + Mobile Network
![Page 20: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/20.jpg)
20
DOAP Architecture
High Distributed Applications
Execution level adaptation mechanisms
Distributed Mobile Application
ISAMadapt (Holoparadigm)
ADAPTATION
User VirtualEnvironment SCHEDULING
Execution Profile Resource Profile User Profile
NamingComuni-cation
Migra-tion
Repli-cation
Interope-rability
Loca-tion
Monito-ring
Java Virtual Machine
Native Operational System
Static Network + Mobile Network
EXEHDA – Execution Environment for High Distributed Applications
![Page 21: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/21.jpg)
21
DOAP Architecture
Distributed Mobile Application
ISAMadapt (Holoparadigm)
ADAPTATION
User VirtualEnvironment SCHEDULING
Execution Profile Resource Profile User Profile
NamingComuni-cation
Migra-tion
Repli-cation
Interope-rability
Loca-tion
Monito-ring
Java Virtual Machine
Native Operational System
Static Network + Mobile Network
Proposals for object:
Migration;Optimized communication;Monitoring
PRIMOS – PRIMitives for Object Scheduling
![Page 22: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/22.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
22
VIC++
For Access Grid
![Page 23: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/23.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
23
VIC ++
• Access Grid– a Grid for people– people interact with Grid resources– interaction between people groups– distributed seminars, discussions, workshops,
classes, ...
![Page 24: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/24.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
24
VIC++
• VIC and RAT– software for video and audio transmission– mbone tools– used by Access Grid (also VRVS)
![Page 25: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/25.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
25
VIC++
• Global Supercomputing 2001– a node at Instituto de Informática - UFRGS
• Experience results– mbone tools are not easy to operate– they lack some features for adaptation to
dynamic environment
![Page 26: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/26.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
26
VIC ++
• VIC ++– an extension of VIC– two main targets:
• an adaptive algorithm– adjust the VIC configurations to the network
conditions– reduce the work of the operator during a session
– to expand the statistics generator of the VIC• in order to provide more data to the researcher
![Page 27: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/27.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
27
VIC++
• Distributed XPDF– alternative to Distributed Power Point
• used by AccessGrid
– working– free software
![Page 28: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/28.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
28
Globus Experience
• Globus installation– Myrinet/PC cluster of II-UFRGS– second semester of 2001– next steps
• use with some academic applications - tests
• evaluate the possibility of integration between our projects and Globus
![Page 29: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/29.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
29
Other Brazilian Grid Projects
• An overview of main Brazilian initiatives– MultiCluster– GridGene– IC Grid– MyGrid
• Probably it is not a complete list • There is also a proposal of a national Grid
![Page 30: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/30.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
30
Other Brazilian Grid Projects
• MultiCluster – II/UFRGS– contact: Prof. Philippe O.A. Navaux
• It aims to efficiently integrate different cluster-based architectures– Myrinet, SCI, Fast Ethernet, …
![Page 31: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/31.jpg)
31
Other Brazilian Grid Projects
• GridGene – UFRJ, LNCC/MCT, UFPB/Campina Grande– contact: Prof. Paulo Bisch
• It aims to support– genome analysis– through development of parallel and distributed software
for Grid environments
• http://www.inf.ufrgs.br/procpar/hetnos/grid/papers/gridgene.pdf
![Page 32: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/32.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
32
Other Brazilian Grid Projects
• IC Grid – UFF– contact: Prof. Vinod Rebello
• Creation of a mini-grid testbed
• Design and development of grid-base middleware
• http://www.ic.uff.br/~vefr/research/cluster.html
![Page 33: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/33.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
33
Other Brazilian Grid Projects
• MyGrid – UFPB– contact: Prof. Walfredo Cirne
• It provides a global execution environment
• It is distributed as a free software• http://lula.dsc.ufpb.br/~walfredo/projetos/
OpenGrid/
![Page 34: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/34.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
34
Other Brazilian Grid Projects
• Proposal: build a national Grid of Cenapads
• Cenapad– HPC national (federal) center
– offers HPC resources to application users
– nowadays there are:• six CENAPADs (HPC centers)
• a SINAPAD (integration of these centers)
• This structure can be the starting point of a national Grid
![Page 35: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/35.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
35
More Grid Projects at Brazil
Brazil / Cenapad Grid
![Page 36: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/36.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
36
Final Remarks
• Several projects on Grid software– adaptive scheduling– Java HP programming– Access Grid/Vic++– Multicluster– Global execution environment– Grid testbed for new Grid middleware
![Page 37: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/37.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
37
Final Remarks
• At least two important Grid applications– HEP – Genome
• Brazilian Grid group
• Sinapad Grid
• Other projects are moving to Grid– mainly cluster (HPC) and distributed projects
![Page 38: Lishep Grid Workshop - February 2002 - UERJ, Rio de Janeiro 1 Regional Center and Grid Development in Brazil Cláudio Geyer geyer@inf.ufrgs.br GPPD - Parallel](https://reader036.vdocuments.us/reader036/viewer/2022062511/55159ba955034674578b5d75/html5/thumbnails/38.jpg)
Lishep Grid Workshop - February 2002 -UERJ, Rio de Janeiro
38
End
A lot of work is necessary ...