“Introduction to the GRID”

Satoshi SekiguchiDirector, Grid Technology Research Center,

Advanced Industrial Science and Technology, Japan

What is the Grid (1)?

The World Wide Web provides seamless access to information that is stored in many millions of different geographical locations

In contrast, the Grid is a new computing infrastructure which provides seamless access to computing power and data distributed over the globe

The name Grid is chosen by analogy with the electric power grid: plug-in to computing power without worrying where and how it comes from, like a toaster

Slide by courtesy of Fabrizio Gagliardi

What is the Grid? (2)

New applications enabled by the coordinated use of geographically distributed resources

e.g., distributed collaboration, data access and analysis, distributed computing

The Grid relies on advanced software, called middleware, which ensures seamless communication between different computers and different parts of the world

e.g., certificate authorities and policies, protocols for resource discovery/access, programming environment

Original motivation, and support, from high-end science and engineering; but has wide-ranging applicability

e.g., from Grand Challenge to Everyday Business

Slide courtesy by Ian and Carl

Who will use the Grid?

Computational scientists & engineers: large scale modeling of complex structures

Experimental scientists: storing and analyzing large data sets

Collaborations: large scale multi-institutional projects

Training & education: virtual learning rooms and laboratories

Environmentalists: climate monitoring and modeling

Corporations: global enterprises and industrial partnership

Slide by courtesy of Fabrizio Gagliardi

Grid – a quick look

No clear definition, changes day by day

GridGrid

Computing Grid（ high speed computing service）

Computing Grid（ high speed computing service）

Data Grid（ File service, DB integration）Data Grid（ File service, DB integration）

PC Grid（Mega computing/ Cycle scavenging）

PC Grid（Mega computing/ Cycle scavenging）

Business Grid（ Data Center Grid）Business Grid（ Data Center Grid）

Grid

Grid

Meta computing

Virtual Computing Center

Grid Data Center

eGov, Business federation

SETI@HOME-like

Bio, High energy, AstroLarge scale data analysis,

Intra Grid

Technical Business future plan

Access Grid, Sensor Grid

Technology Life Cycle (Grid/Electricity)

Research Lab

Technology

Globus, Unicore

Ninf, SETI@homeGrid Testbed

Pilot user

Soft PKGGrid XX

Grid software Product

Business

Utility Data Center

B. Franklin

Volta's Battery Lamp

Power Plant

Today

GridWorld2004

2004 Apr. 27-28　 TFT Hall, Tokyo Japan

The first national scale event for grids in the world

3000 registration, 2000 attended, 20 groups exhibited

GridToday 04

2004 May 26-28　 Philadelphia CC

Exhibit for grid technologies in US

400 attended, 26 groups exhibited

Business with Grids

H/W(Server, Storage, Network)

OS (Linux, Solaris, AIX,…, Windows)

System management tool(Work load, DB, Performance monitoring, Resource management)

Applicatiooon(Stock mgmt, Financial mgmt, CRM, )

Business Flow（ eCommerce, Business Management,…）

Web service system(XML, SOAP, WSDL )

OGSA(Authentication Authorization, Policy, Resource, )

Grid SI/Consulting

constructionService

GlobusToolkit

CommercialSupport

(Consulting)

ASPApplication

Service Provider

Utility Data Center

(Grid Service) (Mediation)

ResourceBroker

PortalDirectoryservice

grid readyserver computer

(Product Sales)

Construction Tools

Grid middleware

grid readyApplication server

grid readymanagement tools

[Y]2004 2005 2006 2007 2008 2009 2010

807

① Grid System ② Grid Service

49 88 12817065 117

171227

[100 Million JPY = 0.9 M USD]

0

300400500600

594

347

529

391

258

700

①

②

Grid

Bu

siness

Grid Market Growth

200100

～～1156

1383

1114

12851191

1100

10741035

0

1200

1500

1600

[Y]2004 2005 2006 2007 2008 2009 2010

2000

1800

1000

1292

1547

1774

2099

12001245

1886

IDCIDC

System – non GridSystem – non Grid

System - GridSystem - Grid

+ Grid service+ Grid service

[100 Million JPY]

987(2002年水準 )

[by NAREGI report]

Economic Impact in IT Market in Japan

Adopt Grid Technology to Business Scene

IT resource Virtualization/ provisioning/Integration of Web servicesIT resource Virtualization/ provisioning/Integration of Web servicesHigher resource utilization usage for saving TCOHigher resource utilization usage for saving TCOAutomate IT resource management in Data centers Automate IT resource management in Data centers Business Continuity, Business opportunityBusiness Continuity, Business opportunity

Science grid Business Scene

HPC Application・ Bio informatics・Weather forecasting, climate simulation・ Structure Analysis

Adopt Grid technology to businessScene・Higher utilization of IT resources•Dynamic load balancing

• Business Application• Web Service• Engineering/Technical…

Global load distributionGlobal IT resource sharing

More complex, higher TCO

Larger in system scale

3-tier model

Buy more servers for sustain performance

More complex heterogeneity

Processor architecture, performance

OS (Windows, Linux, Unix, …)

Non standard APIsDB, AP server, system management

TCO increases significantly

Save TCO by Web service, Grid

Autonomous IT Infrastructure

Autonomous

Dynamic provisioning according to the demand for the resources in case for access spike, system fail.

Virtualizes computing heterogeneity

Fail

Over load　　　　　　　　　

Stand-byServer on duty

M A

E P

M: MonitorA: AnalysisP: PlanE: Execution

Service on demand Pay Per Use (PPU)

1 2 3 4 5 6 7 8 9 10 11 12

Month

Savings

Option – Pay per Use

Basic Charge

Access Spike Overload

Intra Grid for higher resource utilization

Present in-house system Solution by the business grid

Infra mgmt

Personnel Accounting Production CRM CRM

Business Grid M/W

・ TCO saving・more flexibility in business units

EffectSolution

Grid virtualizes IT resource to share

Problem

・ Vertically fixed resource allocation・ less flexibility in dynamic allocation

shared environment

Infra mgmt Infra mgmt

PersonnelAccounting

Production CRMGeneralaffairs

Off loading access spike

Hot spot

Additional resourcemaintain business continuity

network

Online shop A

Settlement

Delivery

Maker P

Maker Q

Online shop B

Parts Maker

storageserver

Additional resource deals with overflow traffic

Guarantee business continuity

EffectSolution

Faster and easier deployment, and provisioning – much less SE work

Problem

unexpected spike-may lose business opportunity

-may lose reliability

Outsourcing IT resources by the grid service provider

Data Center X

Bussiness A

Bussiness B

Outsourcing

Outsourcing

The in-house system of a company<Grid service provider>

Resource offer

Systems can cooperate between the in-house and the data center by the business grid.

EffectSolution

Constructing system using grid middleware.

Subject

Demand of curtailing system possession cost.

Resources are required for load / peak management.

IDC as a backup center in case of calamityDisaster recovery

Business A

Business B

Design SystemDesign System

Accounting systemAccounting system

Personnel systemPersonnel system

AA Business SystemAA Business System

BB Analysis SystemBB Analysis System

Data protection bases

A CorpDB

B CorpDB

C CorpDB

ResourcePool

A CorpA Corp

B CorpB Corp

C CorpC Corp

C CorpC Corp

C CorpC Corp

Pooled resources→A CorpPooled resources→A Corp

Pooled resources→B CorpPooled resources→B Corp

Pooled resourcesPooled resources

Pooled resources (Additional)Pooled resources (Additional)

AA ComputingParts group

CC Business Parts group

BB BusinessParts group

OtherParts group

AP servers (Parts functional bases)

DB serversWeb/AP servers (Business system bases)

Integrated DB bases

Ne

two

rk ba

ses

Load increases

Automatic extension of pooled resources

Restores automatically to pool resources

・ Adding active resources・ Automatic inclusion

(No need for stopping other systems)

Integrated Operation Management Bases

Monitor Monitor

Mo

nito

r

trouble

Automatic extension / self-restoration

Network

Automatic assignment of a zone

TCO curtailmentCommon pool improve utilization ratio

EffectSolutionGrid enables common IT resource pool to shared back up

Subject

Duplicated applications and IT resources

Many unused resources for every system

Realization of optimizing shared IT resources in a group company

Karaoke Grid @ SC2003

National and InternationalGrid Activities

Significant Grid Activities in Asia-PacNAREGI 1,450M JPY ’04 ’03-’07 MW, AP, HW

Business Grid 2,500M JPY ’04 ’03-’05 MW

Asia Grid Initiative

(ApGrid/APAN)

120M JPY ’02-’04 OP

K*Grid 32M USD ’02-’06 HW, MW, AP, OP

NGO approx.

25M SGD

’04-’05 HW, MW< AP, OP

China Grid

Chinese Taipei

EcoGrid AP

Thai Grid ’04- HW, MW, OP

PRAGMA ’02- AP, OP

1USD=110JPY=1,100KrW=33.8TWD=41.3THB=1.69SGD

ApGrid: Asia Pacific Partnership for Grid Computing

Open community for Grid researchers in Asia Pacific

ApGrid:A meeting point for all Grid researchers in Asia-PacificA communication channel to the GGF, and other grid communities (e.g. TeraGrid, UK-eScience, EUGrid, etc.)A computing testbed for making real grid-collaboration

India

ApGrid Testbed – unique features –

Truly (naturally) multi national/political/institutional VO beyond boundaries

Not an application-dedicated testbed – general platformDiversity of languages, culture, policy, interests, …

Grid BYO – Grass roots approachEach institution contributes his resources for sharingNot a single source funded for the development

Physical resourcesMost contributed resources are small-scale clustersNetworking is there, however the bandwidth is not enough

Achievementshave experiences on running international VOverify the feasibility of this approach for the testbed development

Non-profit international Non-profit international consortium established on consortium established on 3 June 1997. 3 June 1997.

Being a high-performance Being a high-performance network for R&D on network for R&D on advanced next generation advanced next generation applications and services. applications and services.

Providing an advanced Providing an advanced networking environment for networking environment for the research and education the research and education community in Asia-Pacific.community in Asia-Pacific.

Promoting global Promoting global collaboration.collaboration.

Tightly collaborating with TransTightly collaborating with TransPAC.PAC.Having meetings twice a year.Having meetings twice a year.

NSF-funded project lead by NSF-funded project lead by UCSD/SDSC. UCSD/SDSC.

11stst workshop was held in March workshop was held in March 2002. 2002.

7th one – Sep.15-17 ‘04

Establish sustained collaborations Establish sustained collaborations and advance the use of the Grid and advance the use of the Grid technologies for applications.technologies for applications.

Expected outcomes:Expected outcomes:Advance scientific applicationsIncrease productive and effective use of the grid by researchers and scientists in the Pacific RimIncrease interoperability of grid middleware in Pacific Rim and throughout the world

Tightly collaborating with ApGrid.Tightly collaborating with ApGrid.Having workshops 2 times a year.Having workshops 2 times a year.

Pacific Rim Application and Grid Middleware

Assembly

March 2004 (catlett@mcs.anl.gov) www.ggf.org

Ed

ucatio

n

Par

tner

ship

s

Global Grid Forum

Specifications and Best Practices

Identify where / what specs are critical

Frameworks / Architectures(how specs fit together)

Research and Technology Directions

Application Requirements

WS/I

W3C

OASIS

DMTF

IETF

GGF must collaborate with (leverage) a

variety of standards organizations

working on underlying

and related

technologies.

Korea: Sangsan Lee (GFAC)Japan: Sekiguchi (GFAC), Matsuoka (GFSG), Itoh (GMAC), and co-chairs, etc

March 2004 (catlett@mcs.anl.gov) www.ggf.org

GGF Meeting Participation

0

100

200

300

400

500

600

GF1 GF3 GF5 GGF2 GGF4 GGF6 GGF8 GGF10

Countries Organizations Group Participants Plenary Participants*GGF3 shows impact of 9-11

TokyoMarch,’03

13th in SeoulMarch,’05

12th in BrusselsSep,19-22 ’04

Roles of the APEC/TEL Grid Workshop

Expected role of APEC/TEL

Government role includes:Encourage basic studiesRemove potential obstacles in advanceEstablish Int’l cooperation framework to speak to other communities, - US, EU, etcNO commitment for each business or project

Business and Science Grid are inseparable from each other:

An ultimate challenge in science grid is always a trailblazer for expanding the business field.

eScience, Large science, Affordable high speed network

Having consensus of shared policy is a MUST for promoting future international grid business opportunity cooperatively,.

security, CA ops, SLA, balance, etc

Grid Challenges

Cope with different management policies of different centersEnsure data security: more is at stake than just money!Guarantee higher reliability for business continuity: prepare for disaster recoveryAgility and resiliency for every business opportunity

Grow rapidly, yet remain reliable for more than a decadeShare data between thousands of scientists with multiple interestsLink major computer centres, not just PCsEnsure all data accessible anywhere, anytime

Business and ENTERPRISEBusiness and ENTERPRISE

SCIENCE and TECHNOLOGYSCIENCE and TECHNOLOGY

Grid TechnologyGrid Technology

Upper MiddleUpper Middle

HPCHPC

Portal/ASPPortal/ASP

Grid Operation CtrGrid Operation Ctr

High speed network (over 1G – 10G)

Ubiquitous GridUbiquitous Grid

IPv6IPv6 SecuritySecurity

Wireless networkTremendous Sensor

Huge datastream

GPSGPS

CamCam

sensorsensor

MobileMobile

Bloodpressure

Bloodpressure

Earth QEarth Q

Database/ExpressionDatabase/Expression

FTTH/xDSL/Wireless AP/100M-FTTH/xDSL/Wireless AP/100M-

DistributedStorage10PB

DistributedStorage10PB

Community InfraCommunity InfraScience and Engineeringplatform

Science and Engineeringplatform

Computing1PFLOPSPC x 100MSC x 10000

Computing1PFLOPSPC x 100MSC x 10000

Peta Scale GridPeta Scale Grid

Nano-techInformatic

s

Nano-techInformatic

s

Access Grid

Access Grid

ChemInfoma

ChemInfoma

Personal Wether s

ervice

Personal Wether s

ervice

Bio-Informatics

Bio-Informatics

MedicalInformatic

s

MedicalInformatic

s

Lower MIddleLower MIddle

P2P gridP2P grid

WebserviceWebservice

eBiz, service

eBiz, service SolutionSolution

UtilityUtility

CommunityCommunity

Server tech

Grid enabled EngineeringGrid enabled Engineering Grid enabled BusinessGrid enabled Business

Grid Architectures and AIST contributions

Grid Fabric

AP partners have Computers: PC, WS, Clusters, Supercomputers; SCE manager;WAN: Internet, APAN, GrangeNet, APII, Tsukuba WAN; Instruments: KEK Accelerators, …

Low Level Grid Middleware

Security, Process, Storage, QoS, etc: Globus, Jxta, CPM, GrACE, OGSA

High Level Grid Middleware

Programming Tools: Ninf-G, Omuni-RPC, Nimrod DPL, MetaCompilerSimulators: Bricks and GridSim, Data farm

Resource Brokers: Nimrod-G

Applications and Portals

ActiveSheet, BioGrid, Drug Design, HEPGrid, ThaiGrid Portal, CFD, Chemical Eng., Environmental and Ecology Preservation,…

Social Framework

Clusters, MPPs,VP, SMPs

Education/training/workshop

Grid RPC/Ninf-GGrid Data Farm

QC Portal

DrugDesign

BioInfoma

Agriculture HEPDG

1 USD = 129 JPY

tools

Summery of my talk

Grid technology has been significantly emerging in science scene.

More demand in business scene.

Grid expands the IT market size and other areas.

Encourage Grid activities in the region is a MUST !

Let’s talk about it for today

Back ups