jens g jensen atlas petabyte store supporting multiple interfaces to mass storage providing tape and...

20
Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Upload: diego-parker

Post on 27-Mar-2015

216 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Supporting Multiple Interfacesto Mass Storage

Providing Tape and Mass Storage to Diverse Scientific Communities

Page 2: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Requirements

Data Management

Locate data

Retrieve data

Page 3: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Requirements

Analyse data on the Grid

Dual access – Grid andnon-Grid

Page 4: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Requirements

CurationLong term storage and archival

Page 5: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Requirements

Very largevolumes

Very largereal-timedata rates

Page 6: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Requirements

Recover data in emergency

= backup

Page 7: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

What is Atlas?

• 1.2 Petabyte capacity• Tape• STK Powderhorn• Interfaces, services

Page 8: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

What is Atlas?

• 1.2 Petabyte capacity• Tape• STK Powderhorn• Interfaces, services• New robot coming in

Page 9: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

We support – Grid protocols

• SRB

– Data management interface

– Metadata, data

• SRM

– Built for very large data volumes

– Very high transfer rates via GridFTP

Page 10: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

What is the Grid?

Distributed computing

Distributed collaborations

and Virtual Organisations

Page 11: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

What is the Grid?

Access is brokered

job

Data is replicated

Page 12: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

What is the Grid?Well defined protocols (sort of)

• File access• Information providers• Job submission• Security

Page 13: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Grid Architecture, SRB

Atlas

SRBScientist

Page 14: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Grid Architecture, SRB

Atlas

LocalSRB

Scientist

Group files into container

Slow network

Fast network

Store container

Remote SRB

Page 15: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Grid Architecture, SRM

Storage Element(SRM)

File Transfer Service

Replica Manager

Replica Catalogue

ApplicationInformation

Services

Components fit togetherto provide Grid servicesAtlas

Page 16: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

We support – non-Grid protocols

TapeDisk cache

vtp, rfio, dcap,…

Page 17: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Who are the customers

GridPP Tier 1

CCLRC facilities

Research councils

e-Science projects

Page 18: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Who are the customers

Small – a few gigabytes

To large – hundreds ofTerabytes

Different customers drivedifferent areas of service

Be all things to all people?

Page 19: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Community

User group meetings

Helpdesk

How to tie the communitytogether?

Page 20: Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities

Jens G Jensen

Atlas Petabyte store

Conclusions

• Supporting multiple communities via multiple interfaces– Grid interfaces and non-Grid– Multiple requirements

• Diversity is good – (up to a point?)– Volume and rates driven by GridPP– Metadata driven by e-Science projects

and RCUK