dpm name server (dpns) namespace authorization location of physical files dpm server requests...

• DPM Name Server (DPNS)• Namespace• Authorization• Location of physical files

• DPM Server• Requests queuing and processing• Space Management

• SRM Servers• v1.1, v2.1 and v2.2

• Database (DPM and DPNS)• MySQL and Oracle are supported

• High scalability and availability• DPM and DPNS scalability is directly linked to the database (MySQL or Oracle) scalability• All servers can be load balanced, except the DPM one

• Security in DPM• GSI• Permissions – Ownership• ACLs • VOMS support• Virtual uids/gids (based on VOMS groups and roles)

SRM v2.2

• Admin• srmPing

• Space Management• srmReserveSpace, srmReleaseSpace, ...

• Permissions and ACLs• srmSetPermission, srmGetPermission, ...

• Transfer• srmPrepareToPut, srmPrepareToGet, ...

• Namespace• srmLs, srmMkdir

The DPM (Disk Pool Manager) is a secure, lightweight and highly scalable system to manage storage on disks

DPM head node

Disk Servers

The Disk Pool Manager (DPM)Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The Disk Pool Manager

Data Management in gLite

Status and Next Steps

file

DPM disk servers

/vo

/dpm/domain

/home

DPM

head node

CLI, C API,

SRM-enabled client, etc.

data transfer

…

• Quotas• srmCopy• NFSv4.1 support• Medical Data Management

• Encryption

hep-service-dpm@cern.ch

New features

http / https

• In the namespace• Secondary groups support

• On disk pools• Secondary groups intentionally not supported at pool level (so that the group or sub-group who actually uses the space gets accounted for it)

• But possibility to:• (1) restrict a pool to a list of groups• (2) add and remove groups later on

• Web access to DPM storage via http and https

The DPMArchitecture

• Largest DPM Storage• 80 TB (Glasgow)

• Largest DPM Name Server• million entries (Oracle)

Th

an

ks t

o G

ido

n M

oo

nt

(Grid

PP

– I

mp

eria

l Co

lleg

e –

Lo

nd

on

)S

ee

htt

p:/

/grid

po

rta

l.he

p.p

h.ic

.ac.

uk/

rtm

/

• DPM target in EGEE• Small & medium-sized sites

• What do sites like?• Easy installation• Easy administration

• DPMs in production• 125 instances• 138 VOs supported

http://twiki.cern.ch/twiki/bin/view/LCG/DataManagementTop(CERN) Akos FROHNER, Jean-Philippe BAUD, David SMITH, Sophie LEMAITRE, Krzysztof NIENARTOWICZ, Lana ABADIE, Rémi MOLLON

(1) dpm-addpool --poolname AtlasPool --def_filesize 1G --group atlas, atlas/Role=production

(2) dpm-modifypool --poolname AtlasPool --group +atlas/Role=lcgadmin,-atlas/Role=production

• GridFTP Server• Secure RFIO Server

• The Disk Servers host the actual physical files• No bottleneck: direct data transfer from/to the Disk Servers

Easy to guarantee space for a given VO

Easy space reservation

• Space can easily be reserved (1) and released (3) byusers (limited)administrators (unlimited)

(1) dpm-reservespace --gspace 200G --group atlas --token_desc atlas_ESD

(2) lcg-cr --vo atlas file:higgs.root --st atlas_ESD –d dpm.cern.ch

(3) dpm-releasespace --token_desc atlas_ESD

• The user space token created in (1) by the DPM administrator can be used by any user in the atlas group. Ex. with lcg_util (2)

$ export DPNS_HOST=dpm.cern.ch$ export DPM_HOST=dpm.cern.ch

$ rfcp file01 /dpm/cern.ch/home/atlas/data/file01$ rfrm /dpm/cern.ch/home/atlas/file01

Usage example: