HAMBURG • ZEUTHEN

DESY Site Report

HEPiX/HEPNTFermilab

2002-10-23Knut Woller

2002-10-23 Knut Woller DESY Site Report IT–Systems 2

HAMBURG • ZEUTHEN

Overview

I will focus on ongoing activities and projects:

Storage and data management

• dCache

• ExaStore

User Registry Project

Windows Migration Project

Mail Consolidation

2002-10-23 Knut Woller DESY Site Report IT–Systems 3

HAMBURG • ZEUTHEN

Storage and Data Management

New requirements and challenges:

Need to decrease storage costs

Increasing number of clients burdens HSM

Distributed clients create awkward data paths, and distributed NFS does not scale

The “Traveling Scientist” requires mobility

Users are increasingly unable or unwilling to judge features or cost of a specific store. They just want to use it.

2002-10-23 Knut Woller DESY Site Report IT–Systems 4

HAMBURG • ZEUTHEN

About dCache

Distributed cache between clients and HSM

Collaborative development at DESY & FNAL

In production use at DESY and FNAL

More labs are looking into it

DESY currently runs about 30TB read pool on IDE RAID servers

All major DESY groups use it by now

For us, it is the method to access HSM data

2002-10-23 Knut Woller DESY Site Report IT–Systems 5

HAMBURG • ZEUTHEN

dCache Features

Allows the use of cheap tape media by largely reducing the number of mounts

Coordinates the site wide data staging and reduces data management manpower

Supports several HSMs (OSM, EnStore, Eurogate)

Can be transparently used by applications through C-API (ROOT supports dCache)

Scales well to thousands of clients and hundreds of pool servers

Can be used in GRIDs (bbFTP, gridFTP)

2002-10-23 Knut Woller DESY Site Report IT–Systems 6

HAMBURG • ZEUTHEN

dCache Development

DESY / FNAL project is well advancedDESY / FNAL project is well advanced

Presentations have been made at recent HEPiX Presentations have been made at recent HEPiX and CHEP conferencesand CHEP conferences

Project information is on Project information is on

http://www-dcache.desy.de

We plan to set up a central read disk pool of We plan to set up a central read disk pool of 100+TB when we migrate to large, cheap tapes 100+TB when we migrate to large, cheap tapes (STK 9940B) in a few months.(STK 9940B) in a few months.

2002-10-23 Knut Woller DESY Site Report IT–Systems 7

HAMBURG • ZEUTHEN

ExaStore

Since 1999, major user groups have demanded a “Large Central File Store” at DESY

Features: Multi-Terabyte, high performance, single filesystem view, random access

AFS will not scale to this size

dCache does not fit the requirements

Commercial NAS solutions do not scale well

EXANET came along in 2000 with a product proposal that suits our needs

2002-10-23 Knut Woller DESY Site Report IT–Systems 8

HAMBURG • ZEUTHEN

About ExaStore

Seen from the outside, the ExaStore is a highly scalable, high performance NAS (or a huge virtual disk)

Internally, it is built from disk and CPU servers and independent RAID arrays. ExaStore’s spice is

• The use of commodity components

• Their cluster file system

• Their redundant server mesh

ExaStore scales in (at least) two dimensions:

• In capacity by adding disks

• In performance by adding nodes and/or uplinks

2002-10-23 Knut Woller DESY Site Report IT–Systems 9

HAMBURG • ZEUTHEN

Why Exastore at DESY?

Because the current jungle of cross mounted NFS disks is an administrative nightmare

Because NFS data management at DESY today is handled decentrally in the user groups. IT wants to fill this gap to make better resource use.

Because scaling the current system of distributed NFS servers reduces stability and manageability

Because current NAS solutions are limited to 12-18TB per box and a fixed number of uplinks and server nodes

Because we do not think it would be wise to invent our own SAN/NAS solution.

2002-10-23 Knut Woller DESY Site Report IT–Systems 10

HAMBURG • ZEUTHEN

ExaStore Experiences

First test system at DESY since April, in beta test since June (4 nodes, 1.5TB)

No crashes in four months

Performance is not yet where we want it to be, but well on the road

We want to acquire a production system with 8 nodes and 12 TB (management approval pending)

2002-10-23 Knut Woller DESY Site Report IT–Systems 11

HAMBURG • ZEUTHEN

User Registry Project

DESY User Registry is old, limited, inflexible

Number of user groups is increasing

Each new complex software system today comes with a proprietary registry (e.g. mailserver, calendar server, Oracle, SAP, …)

Interfaces to HR database, phonebook etc. are required

We need a site wide metadirectory toolbox

Groups have a large demand for delegations of rights

2002-10-23 Knut Woller DESY Site Report IT–Systems 12

HAMBURG • ZEUTHEN

Project Approach

Design phase started in January

We have a clear functional description now

We looked in to commercial (Tivoli, CA, …) and open source (Ganymede) tools, none of which seem to fit our needs

We are gathering troops to start coding

Platform account (unix, windows, kerberos) should be manageable in Q2/2003

Platform adaptors will take some time

2002-10-23 Knut Woller DESY Site Report IT–Systems 13

HAMBURG • ZEUTHEN

Windows Migration Project

The DESY Windows Domain is still NT4

We started rolling out W2K and WXP clients in the DESYNT domain (mostly notebooks)

Basic software support (netinstall) for WXP desktops in DESYNT available this year

Domain servers are NT4, newer ones W2K

.net server look promising, but are not in production use yet

Where possible, we are skipping W2K clients

2002-10-23 Knut Woller DESY Site Report IT–Systems 14

HAMBURG • ZEUTHEN

W2K Migration Status

New project team has been formed within IT

We are finalizing the site wide AD design

New hardware has been / is being acquired

Homedir storage is under reconsideration

We plan to have a working domain in Q1/2003

Migration start foreseen in Q2/2003

DESYNT will stay alive for control systems

2002-10-23 Knut Woller DESY Site Report IT–Systems 15

HAMBURG • ZEUTHEN

Mail Consolidation

We are still in the sad state of supporting sendmail, Exchange, and PMDF

We experience load and capacity problems on all three systems

User ‘requirements’ (real or not) have limited us in the past years

Next step will be mail routing consolidation to get rid of PMDF

We want to end up with one mail router and one mail server solution, both yet unnamed

2002-10-23 Knut Woller DESY Site Report IT–Systems 16

HAMBURG • ZEUTHEN

In General …

… we have been able to increase our IT staff with bright, young colleages (IT is back to 1999 staffing level)

… we start seeing synergy effects by treating windows and unix systems in one group (e.g. Samba, hardware standards)

… we have been able to start a few major efforts and projects

… we are striving for more coherence between Hamburg and Zeuthen

… much of our effort is still required to clean up or legacy from the past (technologically and socially)

… I think we have a few very well working and scalable solutions, e.g. in mass storage (dcache), Linux support, printing

2002-10-23 Knut Woller DESY Site Report IT–Systems 17

HAMBURG • ZEUTHEN

That’s It

Thank you for your attention