scientific data management: an incomplete experimental henp perspective

1D. Olson, SDM-ISIC Mtg, 26 Mar 2002

Scientific Data Management:An Incomplete

Experimental HENP Perspective

D. Olson, LBNL

26 March 2002SDM-ISIC Meeting

Gatlinburg


Particle Physics Data Grid

www.ppdg.net

PI’s:Mount,Livny,Newman

Coordinators:Pordes,

Olson


Contents

• Quick overview of HENP data—Generic data flow—Sizes, timescales—Average physicist view

• What’s hard—Making technology work in production—A clear view for average physicist—Analysis of large datasets—Other things as well

• Today, many issues wrapped in hopes for “Data Grid”


Experimental HENP event data

• Basic character of data is “event”—May be few particles


BaBar event

http://www.slac.stanford.edu/BFROOT/



• Basic character of data is “event”—May be few particles—May be MANY particles


STAR event, Au + Au

www.star.bnl.gov



• Basic character of data is “event”—May be few tracks—May be MANY tracks

• Detector characteristics, beam types, triggers effect the type of events recorded

• Physics analysis is a statistical analysis of many (1000’s, M’s, B’s, T’s) independent events


Generic data flow in HENP

“Skims”, “microDST production”, …

Filtering chosen to make this a convenient size


A collaboration of people

$100M, 10 yr, 100 people

Free?, 10 yr, 20 people

Free?, 1 yr, 10 people, 5x/yr

Free?, 1 mo, 1 person, 50x/yr

(“Typical” example today, LHC is larger)


Example: CMS Tiers


List of major accelerator-based HENP experiments

Experiment Location # physicists Time scale

BaBar SLAC 800 1999 - 2010

STAR BNL / RHIC 450 2000 - 2010

PHENIX BNL / RHIC 450 2000 - 2010

Jlab/CLAS JLAB 200 2000 - 2010

CDF FNAL 800 1995 - 2010

D0 FNAL 800 1995 - 2010

ATLAS CERN 2000 2006 - 2016

CMS CERN 2000 2006 - 2016

ALICE CERN 1200 2007 - 2017

Jlab Hall D JLAB 200 2008 - 2018


Size / frequency of basic activities

Item Size (TB) / Frequency (/yr)

Typical today LHC era (>5 yr)

Raw data 100 TB / yr 1,000 TB / yr

Event Reconstruction

3 / yr 2 / yr

DST data 1 > DST/ raw > 0.1 0.1 > DST/ raw > 0.02

microDST production

0.1 > microDST/DST > .001

?

Physics analysis 10 - 100 * #physicists / year

?


Average physicist view

• Mythology, culture, terminology varies a lot from one experiment to another.

• BaBar— Object view or primary event store (Objectivity)— Event collection objects give primary access points to data

• Event collection has list of references to all event components of interest

• With 100,000 collections, how to organize them?— Ntuples & PAW for final data format, analysis tool

• STAR (first year data, getting started)

— A “production, trigger” is all reconstructed events for a trigger type with a certain version of code, (P00hg, central)

— Access point is list of directory path’s below which all data are stored on disk

— WZ will be setting up STACS— ROOT for data format and analysis tool

• …


What’s hard I, living with technology

• Typical computer center today— A couple STK Powderhorn tape silos, HPSS or home-grown

MSS— 1000 linux processors— Assortment of 100/1000 Gbps network— 50 TB disk (1000 spindles)— Network s/w for I/O (NFS, Objy AMS, RFIO, …)— AFS for distributed collaboration

• Can make large RAID filesystems w/ network access— Faults can affect many nodes

• stale NFS file handles• AFS faults affects nodes across country, work

— Large RAID is $$$• Desire to reduce effect of faults

— Fewer faults— More tolerance

• …


What’s hard II,A clear view for average

physicist

What’s going on in this box?


What’s hard II,A clear view for average

physicist

• What data is available?—“data” means

• List of files? (like STAR)• Collection object w/ pointers to all events? (like BaBar)

—“available” means• On disk? Where?• Exists?

• Does it really have the filters and calibrations I need?

• Is it the “official” version of the data?• …


What’s hard III,Analysis of large datasets

Dataset does not fit on disk, or requires parallel processing, or

is large enough operation that chance of fault is high



• Dataset does not fit on disk—Needs access s/w to couple w/ processing

• SAM, STACS

—Does performance meet demand?


SAM (Sequential data Access via Meta-data)

http://d0db.fnal.gov/sam/


STACS

http://sdm.lbl.gov/projectindividual.php?ProjectID=STACS



• Dataset does not fit on disk—Needs access s/w to couple w/ processing

• SAM, STACS

—Does performance meet demand?

• Needs parallel processing (not very hard)—Can not do analysis on private/personal machine—Schedule access to shared resource (CPU and disk)

• Operation for a single analysis is large enough that faults occur—Need exception handling—Need workflow management to complete failed

tasks or, at least, accurately report status


Example shared nothing cluster

http://www.ihep.ac.cn/~chep01/paper/4-026.pdf


PPDG


Summary

• Faulty technology sets boundary conditions—Fault tolerant will expand boundaries of capabilities

• Data management is coupled with processing—Visualization (access w/o processing) is minor in

HENP—Need access to data when & where it is needed for

processing

• Working on data grid as context for data management

• PPDG has SDM ISIC as one of the technology base projects

scientific data management: an incomplete experimental henp perspective

Documents