xrootd & root considered root workshop saas-fee september 15-18, 2015 andrew hanushevsky, slac

XRootD & ROOT Considered

Root WorkshopSaas-Fee

September 15-18, 2015

Andrew Hanushevsky, SLAC

http://xrootd.org

September 18, 2015 2Root Workshop

Brief History of XRootD

1997 – Objectivity, Inc. collaboration Design & Development to scale Objectivity/DB

First attempt to use commercial DB for Physics data Successful but problematical

2001 – BaBar decides to use root framework vs Objectivity Collaboration with INFN, Padova & SLAC created

Design & develop high performance data access system Work based on what we learned with Objectivity

2003 – First deployment of XRootD system at SLAC2015 – Wide deployment with several implementations ALICE, ATLAS, CMS, EXO, Fermi, LSST; among others Protocol available in dCache, DPM, and EOS


Why the Name XRootD?

Spurred by the pre-packaged rootd daemon ROOT team planned to upgrade it

We wanted to avoid duplication Easier to get people to use a familiar

name We tried rebranding in 2008

Structured Cluster Architecture for Low Latency Access (SCALLA)

Total failure!


ROOT Related Enhancements

Retrospectively, the two important ones… Server-Side

Vector Reads Client-Side

Cross protocol redirects


Vector Reads

Developed by Rene Brun’s team Work added to XRootD but with a twist!

Can read blocks from multiple files at once Feature never used by ROOT I/O

Probably never will be used In some ways we wish we had not added the twist

All that said… Many applications still don’t use vector reads!

Due to not using TTreeCache• And it’s been there for a very long time


Vector Reads Enhancements

Enhanced over the years Storage system level vector reads

added Prior code unrolled vector reads before

sending the request to the storage system Developed by Brian Bockelman, UNL

Vector read proxy pass-through Proxy server performance improvement


Cross Protocol Redirects

Developed by Lukasz Janyst Allows ROOT I/O to switch protocols

Can improve worker node clustered storage I/O

E.G. xroot://->file:// Can support multi-protocol federations

E.G. xroot:// -> http://

Currently only the xroot client can do this But it’s simple for any protocol to do so

We hope all multi-protocol capable plug-ins will add it

Available in ROOT 6.04.x


ROOT I/O & XRootD I/O

XRootD can only do so much It already efficiently handles most

ROOT I/O

ROOT I/O adopted many tuning knobs Optimize baskets, autoflush, learning,

TTreeCache, streaming, spliting, etc It’s complicated and not easy to

optimize Once optimized it’s specific to an analysis

mode Change the mode and you may be in trouble!


Why All The Optimizations?

It’s mostly due to spinning disks! File organization & I/O requests are very

sensitive to high latency devices

It’s also due to network I/O Small reads suffer high latency

Especially on the WAN

Network issues will be always with us Unless “entangled” networks come to

pass


Are We Too Focused on HDD?

Low HDD $/TB prices are still dropping But not in a good way!

HDD is becoming more dense, so … Cost per terabyte is vastly decreasing! But HDD’s are not getting any faster

They are becoming slower in many respects Shingled drives come to mind

We may be too mesmerized by the “deal”


Need a Change in Hardware!


The Cost Factor Is Still Large

This is what EMC thinks But the ratio is decreasing somewhat

faster


HDD Apples to SSD Oranges!

HD’s getting more dense but not faster Approaching archive use utility!

HD’s use more power and emit more heat Electricity is not getting cheaper


The ROOT I/O SSD Challenge

If by 2018 SSD’s become active storage Either in a hierarchy or primary storage

ROOT I/O may be insufficient Object layout & access algorithms HD-

oriented SSD’s have their own peculiarities

For example, large page read-out size

Time to start rethinking ROOT I/O! How to get the most out of SSD’s


XRootD & SSD

XRootD already is SSD ready Already supports tiered storage (i.e.

SSD+HDD) Used by SLAC for the ATLAS Tier 2

Reasonable approach until SSD prices drop HD rival is estimated by 2020

Based on improvements to 3D NAND technology

So, 2020 may mean primarily SSD access Will ROOT IO be ready?


An Orthogonal Approach

Developing a Scalable Service Interface Leverage XRootD features for services

E.G. Clustered mySQL for LSST queries Testing a 150 node cluster at scale!

SSI runs in parallel with XRootD data service

It’s an optional plug-in for increased flexibility

What is it? Essentially a LORB (Lightweight Object Request

Broker)


The 10,000 KM LORB View

XrdSsiGetClientService()

XrdSsiServiceProvision()

XrdSsiSessionProcessRequest()

XrdSsiRequestProcessResponse()

Client-SideProcessing Flow

Three objects (simplicity) Service, Session, and Request objectsClient changes object Reflected at the serverServer changes object Reflected at the clientObject Brokering Uses XRootD technology Client redirected to the server than can handle the object (provision)SSI objects and interactions have been abstracted to be compatible with most otherprotocols

XrdSsiGetServerService()

XrdSsiServiceProvision()

Server-SideProcessing Flow

XrdSsiResponderSetResponse()

XrdSsiSessionProcessRequest()


What Can You Do With This?

Implement arbitrary clustered services Provide ROOT IO flexibility

I/O can be optimized out of channel Dependent on what the client is doing

Provide PROOF a lot more flexibility Optimized event delivery

The possibilities are endless LSST is at the forefront in leveraging

SSI Necessary given their real-time

requirements


Conclusion

XRootD is under active development Always looking for new ideas

Feel free to suggest them Be a contributor – it’s open source

Available on github! Consider joining the XRootD

collaboration It costs no money to join

See more at http://xrootd.org/


AcknowledgementsCurrent Software Contributors ATLAS: Doug Benjamin, Ilija Vukotic CERN: Andreas Peters, Sebastien Ponce, Michal Simon, Elvin

Sindrilaru Fermi: Tony Johnson Root: Gerri Ganis SLAC: Andrew Hanushevsky, Wilko Kroeger, Daniel Wang, Wei

Yang UCSD: Alja Mrak-Tadel , Matevz Tadel UNL: Brian Bockelman WLCG: Mattias Ellert, Fabrizio Furano, David Smith

US Department of Energy Contract DE-AC02-76SF00515 with Stanford University

xrootd & root considered root workshop saas-fee september 15-18, 2015 andrew hanushevsky, slac

Documents