2

Grid Computing

Presented by

Mohammad Reza Gerami gerami@ipm.ir

Grid Computing 3

Outline

• Conception

• Basic Issues

• Grid Architecture

• Standards for Grid Environments

• Key Components

• Applications

The Grid dream is to allow users to tap into resources off the Internet as easily as electrical power can be

drawn from a wall socket

Grid Dream

4 Grid Computing

Compute Grids

Parallelize logic execution

Data Grids

Parallelize data storage

Grid Computing =

Data Grids + Compute Grids

Data Partitioning + Affinity Map/Reduce

What is Grid Computing?

5 Grid Computing

Grid Computing 6

What is Grid Computing?

• Allows sharing and coordinated use of diverse resources in dynamic, distributed “virtual organizations”.

Why Grid Computing?

The idea of Grid computing resulted from the confluence of three developments:

The proliferation of largely unused computing resources (especially desktop computers)

Their greatly increased CPU speed in recent years

The widespread availability of fast, universal network connections (the Internet).77

7 Grid Computing

High performance computers (formerly called supercomputers) are very expensive to buy and maintain.

Much of the enhancement of computing power recently has come through the application of multiple cpus to a problem (e.g., NCSC had a 720 processor IBM parallel computer).

Many computing tasks relegated to these (especially massively parallel) computers could be performed by a ―divide and conquer‖ strategy using many more, although slower, processors as are available on a Grid.

Why Grid Computing?

8 Grid Computing

Grid Computing 10

Electrical Power Grid Analogy

Electrical power grid

• users (or electrical appliances) get access to electricity through wall sockets with no care or consideration for where or how the electricity is actually generated.

• “The power grid” links together power plants of many different kinds

The Grid • users (or client applications) gain

access to computing resources (processors, storage, data, applications, and so on) as needed with little or no knowledge of where those resources are located or what the underlying technologies, hardware, operating system, and so on are

• "the Grid" links together computing resources (PCs, workstations, servers, storage elements) and provides the mechanism needed to access them.

Grid Computing 11

Why need Grid Computing?

• Core networking technology now accelerates at a much faster rate than advances in microprocessor

speeds • Exploiting under utilized resources

• Parallel CPU capacity

• Virtual resources and virtual organizations for collaboration

• Access to additional resources

Grid Computing 12

Who needs Grid Computing?

• Not just computer scientists…

• scientists “hit the wall” when faced with situations: – The amount of data they need is huge and the data is

stored in different institutions.

– The amount of similar calculations the scientist has to do is huge.

• Other areas: – Government

– Business

– Education

– Industrial design

– ……

Grid Computing 13

Types of resources

• Computation

• Storage

• Communications

• Software and licenses

• Special equipment, capacities, architectures, and policies

Grid Computing 14

Job Scheduling

Grid Computing 15

Security

• Access policy - What is shared? Who is allowed to share? When can sharing occur?

• Authentication - How do you identify a user or resource?

• Authorization -How do you determine whether a certain operation is consistent with the rules?

Grid Computing 16

Grid Security Model

Grid Computing 17

Grid User Roles ---A User’s Perspective

• Enrolling and installing grid software

• Logging onto the grid

• Queries and submitting jobs

• Data configuration

• Monitoring progress and recovery

• Reserving resources

Grid Computing 18

Grid User Roles ---An Administrator’s Perspective

• Planning

• Installation

• Managing enrollment of donors and users

• Certificate authority

• Resource management

• Data sharing

Grid Computing 19

Grid Architecture

Application

Collective

Resource

Connectivity

Fabric

Application

Transport

Internet

Link

GRID Internet

Grid Computing 20

Grid Architecture

• Fabric layer: Provides the resources to which shared access is mediated by Grid protocols.

• Connectivity layer: Defines the core communication and authentication protocols required for grid-specific network functions.

• Resource layer: Defines protocols, APIs, and SDKs for secure negotiations, initiation, monitoring control, accounting and payment of sharing operations on individual resources.

• Collective Layer: Contains protocols and services that capture interactions among a collection of resources.

• Application Layer: These are user applications that operate within VO environment.

Grid Computing 21

Key Components

• Portal/user interface

Grid Computing 22

Key Components

• Security

–Grid Security Infrastructure (GSI)

Grid Computing 23

Key Components

• Broker

–Monitoring and Discovery Service (MDS)

Grid Computing 24

Key Components

• Scheduler

Grid Computing 25

Key Components

• Data management

– Grid Access to Secondary Storage (GASS)

Grid Computing 26

Key Components

• Job and resource management

– Grid Resource Allocation Manager (GRAM)

Grid Computing 27

Applications • The Southern California Earthquake Center uses

Globus software to visualize earthquake simulation data.

Grid Computing 28

Applications • Scientists in the Earth System Grid (ESG) are producing,

archiving, and providing access to climate data that advances our understanding of global climate change. ESG uses Globus software for security, data movement, and

system monitoring.

Grid Computing 29

Applications

• Computational scientists at Brown University are using the Globus Toolkit and MPICH-G2 to simulate the flow of blood through human arteries.

Grid Computing 30

Applications • Globus Toolkit-driven Grid computing is central to

management of large datasets generated by colliders such as those at CERN. This simulation shows two colliding lead ions just after impact, with quarks in red, blue, and green and hadrons in white.

Grid Computing 31

Software implementations and middleware

• Advanced Resource Connector (NorduGrid's ARC)

• Berkeley Open Infrastructure for Network Computing (BOINC)

• Globus Toolkit

• Sun Grid Engine

• ProActive

• UNICORE

• SDSC Storage resource broker (data grid)

• gLite (EGEE)

• NInf GridRPC

• IceGrid

• Java CoG Kit

• Alchemi

• GridGain

• gridGISTICS

• Gridbus Middleware

• Vishwa

• UGP

• GRIA

Grid Computing 32

Organizations & Production Grids

Alliances and organizations

• Open Grid Forum (Formerly Global Grid Forum)

• Object Management Group

Production grids

• Enabling Grids for E-sciencE

• NorduGrid

• Open Science Grid

• OurGrid

• Sun Grid

• Xgrid

• Distributed European Infrastructure for Supercomputing Applications DEISA

• FusionGrid

Grid Computing 33

Grid Projects International Grid Projects • Open Middleware Infrastructure Institute Europe (OMII-Europe) -

May 2006 -> May 2008

• Enabling Grids for E-sciencE (EGEE) - March 2004 -> March 2006

• Enabling Grids for E-sciencE II (EGEE II) - April 2006 -> April 2008

• BREIN — September 2007 → August 2009

• DataTAG - January 2001 -> January 2003

• European DataGrid (EDG) - March 2001 -> March 2004

• BalticGrid - November 2005 -> April 2008

National Grid Projects • D-Grid (German)

• GARUDA (Indian)

• grid computing project at VECC (Calcutta, India)

• China Grid Project

• INFN Grid (Italian)

• KnowledgeGrid Malaysia

• NAREGI Project

• Singapore National Grid Project

• Thai National Grid Project

• BELNET Grid, Belgium

Grid Computing 34

Prospect of Grid computing

The Grid aims ultimately to turn the global network of computers into one vast

computational resource.

Related to many areas in computer science

Being developed by hundreds of researchers and software engineers around the world.

The goal of IRAN-GRID project is to provide: uniform, integrated computational

platform/infrastructure based on the existing computational clusters located at major Universities and Research

establishments.

During the first phase of this project: 12 Universities and Research establishments connected via IPM-GCG clusters.

What is Iran National Grid?

35 Grid Computing

Grid Computing 36

مامریت

داوشگا ي مراکس تحقیقاتی در سطح کشر برای تی 12اتصال • تان محاسباتی باال

:استان تران پژطگا داص اي بيادي• داطگا صعتي اهيركبير• داطگا صعتي ضريف• داطگا تراى• داطگا علن صعت ايراى• داطگا تربيت هدرس•

:بروام آتی داطگا ضيد بطتي• داطگا ضاد• پژطگا افضا• پژطگا ضيخ بايي• هسس اقياس ضاسي•

:استان اصفان داطگا صعتي اصفاى•

:بروام آتی داطگا اصفاى•

:استان فارس داطگا صعتي ضيراز• داطگا ضيراز•

:استان خراسان رضی داطگا فردسي هطد•

:استان مرکسی داطگا تفرش •

:استان قسيیه داطگا بيي الوللي اهام خويي•

:استان کرمان پارك علن في آري كرهاى •

:استان کرماوشا داطگا رازي كرهاطا•

:بروام آتی

:استان زوجان داطگا زجاى•

:استان آذربایجان شرقی داطگا تبريس•

:استان مدان داطگا بعلي سيا•

:استان سمىان داطگا ضارد•

:استان سیستان ي بلچستان داطگا سيستاى بلچستاى•

نقاط متصل به تورين

تي تورين برنامه آ

EGI

This platform will enable and support e-Science in Iran by offering:

high performance computing (HPC)

high-throughput computing (HTC)

data storage capacity needed for very large scale computational projects

What is Iran National Grid?

39 Grid Computing

User Interface (UI) [ui.ipm.ac.ir]

Resource Broker or Workload Management Service (RB, WMS) [wms.ipm.ac.ir]

Information System (top-bdii) [top-bdii.ipm.ac.ir]

Virtual Organization Membership Server (VOMS) [voms.ipm.ac.ir]

Logical File Catalog (LFC) [lfc.ipm.ac.ir]

Monitoring and accounting [monbox.ipm.ac.ir]

IPM-grid Global-Level Architecture

40

The term "grid computing" suggests a computing paradigm similar to an electric power grid - a variety of resources contribute power into a shared "pool" for many consumers to access on an as-needed basis

Ideally the user does not know or care where the computing operation is being performed; the process is invisible to the user.

Middleware handles security, authentication, authorization, resource selection and routing of input and output seamlessly.

How to use Grid?

41

CERN

SETI@home

DNet (distributed.net)

GRID.ORG (anti-cancer ligand screening)

IBM Smallpox cure

Entropia.org

…

Examples of Grid Projects

42

Registration

43

Registration

44

In grid computing a process is called a job.

Users submit jobs to the system via one of the participating institutions

A user of Grid computing does not need to have the data and the software on the same computer, and neither must be on the user’s home (login) computer.

Job Management

45

There are 3 prerequisites for starting job submission:

Prerequisites

Receiving Your

Certificate

Signing for a VO

Obtaining Access to

UI

Receiving Your

Certificate

Signing for a VO

Obtaining Access to

UI

46

Connecting to the UI

Installing the

certificate

Creating a proxy

Running a job

Job Submission

Connecting to the UI

Installing the

certificate

Creating a proxy

Nightingale$ ssh gerami@ui.ipm.ac.ir -p 4422

$mkdir .globus $mv YourCert .p12 .globus/ $cd .globus/ $openssl pkcs12 –nocerts –in *.p12 –out userkey.pem $openssl pkcs12 –clcerts –nokeys –in *.p12 -out usercert.pem

$voms-proxy-init -voms ipmgrid $voms-proxy-info –all

47

Job Submission

Connecting to the UI

Installing the

certificate

Creating a proxy

Running a job

Running a job

Type="Job"; Executable = "/bin/hostname"; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out","std.err"};

$ glite-wms-job-submit -a -r Resource hostname.jdl $ glite-wms-job-status MyJobID $ glite-wms-job-output - - dir output MyJobId

48

grid.ipm.ir

Grid Computing 49

grid.ipm.ir

Grid Computing 50

grid.ipm.ir

Grid Computing 51

grid.ipm.ir

Grid Computing 52

Grid Computing 53

Reference • ―The Anatomy of the Grid(Enabling Scalable Virtual

Organizations)‖

---by Ian Foster, Carl Kesselman, Steven Tuecke

• ―Physiology of the Grid (An Open Grid Services Architecture for Distributed Systems Integration )‖

---by Ian Foster, Carl Kesselman, Jeffrey M. Nick, Steven Tuecke

• ―Grid Computing:Past,Present, and Future‖--- by Elias Kourpas, June 2006

• ―Introduction to Grid Computing‖ ---IBM Redbook,2005

• IBM Grid Computing:www-03.ibm.com/grid/index.shtml

• Globus website: www.globus.org

54

Thank You

Question?