o.büchner, fzj/zam 1 core d-grid infrastructure astrogrid-d meeting 24./25.07.2006 ari heidelberg

O.Büchner, FZJ/ZAM 1

Core D-Grid Infrastructure

AstroGrid-D Meeting

24./25.07.2006

ARI Heidelberg

O.Büchner, FZJ/ZAM, 24.07.2006 AstroGrid-D Meeting 2

Topics of the Infrastructure Project DGI

FG 1: D-Grid base software (Univ. Dortmund) Globus, Unicore, LCG, GridSphere, GAT Large Storage, Data-Interface VO, Management, Tools

FG 2: Installation and operation of the infrastructure Installation and integration of the Core D-Grid Operation and support infrastructure Ressourcen description and benchmark Monitoring, accounting, billing

FG 3: Network and security (DFN) Transport protocols, VPN AAI, firewalls, CERT

FG 4: D-Grid project management (FZK) Project management Sustainable buisiness models

(FZJ)

(Uni Hannover)

Grid software closeto applications

Communication

Operation of the infrastructure

Grid software closeto resources

Coordination


Partners of the Core D-Grid

PC²

RRZN

TUD

RZG

LRZ

RWTH

FZJ

FZK

FHG/ITWM

Uni-KA

Site Resource Amount

FZJ/ZAM IBM Supercomputer with 8,5 TFlopsSTK data robot system with 2,8 PByte

32 CPUs300 TByte

FZK/IWR 8 nodes Opteron 2x2.2 GHz 8 processors of a system NEC SX-5 1 p630 with 4 processors1 SX-6i to do tests2 nodes Opteron 2x2.2 GHz to do tests

100%50%50%50%50%

LRZ SGI high performance system with 20 TFlop/sIntel IA32 and IA 64 Cluster, IBM p690, SunFire 80

5%5% 5%

MPI/RZG IBM supercomputer with 4,5 TFlops, PC cluster with 2 TFlopsData robot system with 8 PByte

32 CPUs400 TByte

PC² Cluster of 400 Xeon 64 Bit processors, high performance visualization and FPGAs

10%

RWTH/RZ 2 SunFire 6900 with 24 UltraSPARC IV each 100%

TU-Dresden/ZIH

SGI O2K(56 proc)/O3K(192 proc.) : T3E (64 proc):PC cluster with 30 processors,end off 2005: new system with 1000 proc.

10%20%20%2%

Uni-H/RRZN PC-Cluster mit 64 CPUs assoc.

Uni-KA PC-Pool assoc.

FHG/ITWM assoc.


Nutzer

Applicationdevelopment

anduser access

GAT API

higherGrid

Functions

Baseservices

distributeddata archives

User/Application

Network-infrastructure

LCG/gLite

Globus

AccountingBilling

User/VO-Mngt

SchedulingWorkflow Management

Datenmanagement

Security

Plug-In

UNICORE

distributed compute

Resources

GridSphere

Monitoring

Data/Software

Uniform

Integration

methods

standardizedservices / interfaces

Integration of the resources into the Core D-Grid


Core D-Grid – state of the work

LCG/gLite FZK

Globus LRZ, FZK, TU-Dresden, (FZJ)

UNICORE FZJ, RWTH, LRZ, RZG, TU-Dresden, PC2

dCache FZJ

Operational middleware installations:

Operation of the infrastructure (under development):

Registration authorities

VO and user management

User support (helpdesk)

Point of Information


IWarp 6x Dualxeon32

OpusIB

64x DualOpteron

InfiniBand

Internet

AFS

external UI

Information Index

Resource Broker

host=dgrid-rb.fzk.de

VOMS

host=dgrid-voms.fzk.de

other sites

CE SE

DESY

User Interface

File Catalog

Storage Element

Computing Element

Computing Element

local disk

host=dgrid-rls.fzk.de

host=dgrid-se.fzk.de

host=dgrid-ui.fzk.de

ssh

host=dgrid-ce.fzk.de

User

ports=2811 (GridFTP),20000-25000(Globus Port Range - GPR)

ports=2170,77729000-9002,2811, 20000-25000

one port per VO:dgtest=15000 ports=5010,8085

ports=2119,2811

20000-25000

LCG Services in Karlsruhe


Globus configuration

Globus client

Firewalls

Computeresource

FZK

Computeresource

ZIH

Computeresource

LRZ

GSIgrid-proxy-init

GridFTPglobus-url-copyGRAM

globus-job-runglobusrun-ws MDS4

wsrf-query


UNICORE configuration

Unicore clients

Firewalls

Unicore Gateway

NJS

FZJ/JUMP

Computeresource

JUMP

NJS NJS

……

usite=dgrid

vsite vsite

Unicore Gateway

2nd identical gatewayat FZK

Port 4009

Port 4009Port 8181

TSI


Status of the UNICORE installations

Vsite # CPU Peak performance [TFlops]

FZJ_JUMP 32 Power 4+ (IBM p690) 0.218

RZ_RWTH-AACHEN 48 Sparc / Opteron 0.23

ZIH_A3K 192 Itanium (SGI Altix) 1.1

ZIH_O3K 128 MIPS (SGI Origin) 0.102

ZIH_O3K2 64 MIPS (SGI Origin) 0.051

lrz-itaniumcluster 134 Itanium 0.86

lrz-migrationscluster 32 Itanium (SGI Altix) 0.41

lrz-pentiumcluster 138 Pentium 0.80

rzg-sp5 20 Power 5 (IBM p575) 0.152

pc2-arminiuscluster 416 Xeon / Opteron 2.6


Cache

DataPool

Server…

Supercomputer JUMP

UI

TSMStorageBackend

DataPool

Server

Cache

SRM/Admin

Server

dCache

GPFSCommon Filesystem

Blue Gene/L

JUBL

Firewall

DMZ

host=zam420 host=zam213 host=zam214

host=zam452

VO=ILDG, DGI

gridftphttps, etc

domain=.zam.kfa-juelich.de

Storage Element in Jülich

O.Büchnerr, FZJ/ZAM, 24.07.2006 AstroGrid-D Meeting 11

Public key infrastructure

Grid certificates by DFN or FZK

Registration authorities at partner sites

FZJ/ZAM, FZK/IWR, LRZ, RWTH/RZ, RZG/MPG, PC2, TUD/ZIH

RAs of DFN

RAs of FZK

Registration

Multi-community ressources

3-level hierarchy

Community → VO → VO member

Interface to VOMS

Core D-Grid access to Compute Resources

O.Büchnerr, FZJ/ZAM, 24.07.2006 AstroGrid-D Meeting 12



Community VO VO-Speaker Resource requested approved Status

Kern-D-Grid administrator W. Gürich Jump (FZJ) 555 0 requested

Astro-Grid-D sun S. Sonne SunFire (AC) 20 15 approved

Kern-D-Grid trial Ch. Dohmen Opteron (FZK) 300 0 rejected

Medi-Grid mri M. Müller Xeon (FZK) 266 266 approved

…



Installation of a Support Infrastructure

Community A Consulting Center Community A

Consulting Center Community A Consulting Center Community

Consulting Center

Dienste-AnbieterDienste-

AnbieterDienste-AnbieterService

provider

Usersupport

Operationmngt.

DGUS

D-Grid User Supportcommon by FZK and FZJ

Entwickler JEntwickler J

Developer

Deployment-Team

Communities DGI

common

trouble ticket

system or

compatible

systems


D-Grid User Support

Community Consulting Center

D-Grid User Support

Service / middleware administrator

Resource-Provider

User requests


D-Grid User Support


D-Grid User Support

D-Grid User Support

dgus.d-grid.de Generation of trouble tickets

User support e-mail addresses of partners

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

o.büchner, fzj/zam 1 core d-grid infrastructure astrogrid-d meeting 24./25.07.2006 ari heidelberg

Documents

core dgrid slide

core dgrid operation

dgrid base software

core dgrid state

infrastructure installation

fzj unicore fzj

tflops data robot system

infrastructure project