alberto colla - cern alice off-line week 1 alberto colla alice off-line week cern, may 31, 2005...

Alberto Colla - CERNALICE off-line week 1

Alberto Colla ALICE off-line weekCern, May 31, 2005

Table of contents:

● Summary of requirements

● Description of the DB access classes

● Some working examples

● Comments/Remarks

● Work in progress

AliRoot DataBase access classes


• simple user interface (data accessible by strings & run numbers)

• Distributed file system as the final storage/retrieval environment

• Possibility to dump data locally• possibility to retrieve data in parallel from different sources (local files + Grid)

• technique used to store/retrieve object should be transparent to user

• automatic versioning (objects will be read-only)

• run-dependent objects ( retrieval handled by a RunLoader)

• Underlying framework: ROOT/AliRoot + UNIX/Grid

Summary of requirements


“History” of DB access classes

• Original idea and first implementation of DB access classes by T. Kuhr (late 2004)

• Since February 2005: work on the framework implementation is carried out in the core offline group

we are receiving a lot of comments and suggestions from detector experts!


DB access classes schema

AliRunData

DB object

AliObjectMetaData

AliRunDataStorage

AliRunDataFile

AliRunDataOganizedFile

AliRunDataAlien

Local file

Loacl file in a defined folder structure

AliEn file

AliRunLoader

(AliSelectionMetaData)

User code

GetRunObject

Get/PutGetEntry/PutEntry

Note: classes names are under discussion and are subject to change!


AliRunData

• Container for a basic databse object

• Contains:● The object● the object’s MetaData: a set of information fully describing the object

• Something on the database objects:

• It is detector specific● can be any object inheriting from TObject● each AliRunData object contains a single object (which can be a container of more elementary objects)● the validity of the object is specified by a RUN range set in the object’s metadata


AliMetaData and derived classes

AliMetaData

AliOBjectMetaData AliSelectionMetaData

Base class, contains methods to compare different AliMetaData objects, etc...

Contains a set of information which fully describes the object

Contains the subset of information used to retrieve the object


AliObjectMetaData

• Each object is associated to its “AliObjectMetaData” which fully

describes it

• pieces of information (AliObjectMetaData data members) constituting

the object’s MetaData:- const char *fName: see later

- int fFirstRun, int fLastRun: validity range of the object

- int fPeriod: LHC fill Period

- const char *fFormat: object`s format (TTree structure, histo structure, etc...)

- const char *fResponsible: name of the person `responsible´ of the object

- const char *fExtraInfo: free field (anything else you may want to know)

- int version: automatically set during storage

• ideas and suggestions on the metadata descriptors are being collected and th efinal

list will change


AliSelectionMetaData

• Only a subsample of all the ObjectMetaData entries are used for data retrieval:

- name- Run validity range

(firstRun - lastRun)- version

• AliSelectionMetaData is used uniquely during data retrieval, to specify a series of selection criteria:

`For the object called “name” and for all runs from firstRun to lastRun, I want this version´

AliSelectionMetaData class


MetaData name

• The metadata name must have a simple (and fixed) structure in

order to ease the retrieval:

Detector/DBType/DetSpecType

Detector´s name(ZDC, TPC...)

DataBase type(Calib, Align...)

Detector specific type of the object (Pedestals, Gains, DeadChanMaps ...)

• Example: “ZDC/Calib/Pedestals”

• In the current implementation this structure is fixed (AliMetaData complains if it is different)


AliRunDataStorage class

• Base class, provides a virtual interface to the database

• Singleton (only one instance of AliRunDataStorage is allowed at any time)Many ideas have been gathered on the “singleton issue” and on the possibility

to retrieve objects from different sources (see Magali’s talk). We are discussing about it…

• Storage and retrieval of objects will be performed through AliRunDataStorage

public methods:

• Bool_t Put(const TObject* object, const AliObjectMetaData& objMetaData)

• const TObject* Get(const char* name, Int_t runNumber)

• The Get function is also called by AliRunLoader::GetRunObject(const char *name)

method (the RunLoader knows the run number)

• Implementation of storage/retrieval of object is performed by the active instance of AliRunDataStorage.


AliRunDataStorage class (2)

• Two other public methods of AliRunDataStorage:

• Bool_t RecordToFile(const char *filename=“DB.root”)Prepares for recording into a local file the entries (AliRunData objects) as they are retrieved from the database

• void Select(const AliSelectionMetaData& selMetaData)specifies one or more selection criteria used for data retrieval

• Examples of the functionality of these two methods will be given later


AliRunDataFile

• Access class for a locally stored object• inherits from AliRunDataStorage• allows one or more objects to be stored in a single Root file• Structure of a stored object:

DBFile.root:

TKey: AliRunData “name1;1”TKey: AliRunData “name1;2”TKey: AliRunData “name2;1”...

Key cycle (beware, this is not the object‘s version!)

“name” is defined by the object‘s MetaData; if name contains slashes (´/´) a TDirectory structure is created

• constructor: AliRunDataFile(const char *filename=“DB.root”, Bool_t readOnly=kTRUE)

• probably not the most suitable way to store large amounts of data; this class is

useful for “local” work on the objects by sub-detector experts


AliRunDataOrganizedFile

• Access class for a object stored into a local file inside a well defined directory structure• inherits from AliRunDataStorage• each root file contains one single object • Structure of a stored object:

DBFolder

Object‘s name1

Object‘s name 2

...

Run1-10_v0.rootRun11-20_v0.rootRun21_v0.rootRun22-30_v0.rootRun22-40_v1.root...

Single AliRunDataentry


AliRunDataOrganizedFile (2)

• The only data member of AliRunDataOrganizedFile class is the folder name (default: “$ALICE_ROOT/DB” )

• When an object is stored with AliRunDataOrganizedFile:

– Its location is determined by MetaData's name (if a folder does not exist itis created)

– The file name is determined by the run validity set in the MetaData;version number is automatically set

– Note: Version is incremented also in case the run validity range of the object “overlaps“ the one of an already existing object!

– File name sintax: Run#firstRun-#lastRun_v#Vers.root orRun#Run_v#Vers.root

– The run validity and version of the stored object's MetaData areautomatically set to -1, to avoid inconsistencies if the file name ischanged by hand– When retrieved, the run validity and version of the object's MetaData arereset according to the file name


AliRunDataAlien

• access classes to a object stored into a Grid file

• inherits from AliRunDataStorage

• not yet implemented – will be based on the ROOT TGrid/TAlien plugin

• the file structure will be probably similar to that built with the

AliRunDataOrganizedFile class (one object per file)


Two examples of calibration objects

1. $ALICE_ROOT/ZDC/AliZDCCalibData class:

• container for ZDC calibration data

• Data members: an array of pedestal values (floats)

an array of energy calibration coefficients (floats)

2. Calibration object for SPD (ITS):

• Under development, see Paul Nilsson´s talk

• Preview: map(s) of SPD dead and noisy channels• object: (array of) TClonesArray(s)

These are two “simple” calibration objects:

• object size will be small (few Kbytes)

• update frequency will be low (< 1/day, say) constant within a run


Usage of the AliRoot DB access classes

1. Storing data

First: specify the object's MetaData

Suppose we want to store a set of ZDC pedestals (set in the object AliZDCCalibData* zdcdata) which we'll use to calibrate runs 10 to 100

AliObjectMetaData md(“ZDC/Calib/Pedestals”,10,100,...);

Run validityName

Second: Create an AliRunDataStorage instance (e.g. an AliRunDataOrganizedFile)

AliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);

Third: Store the object with AliRunDataStorage::Put method

dof->Put(zdcdata, md);

The object is written in file: DBFolder/ZDC/Calib/Pedestals/Run10-100_v0.root


Usage of the AliRoot DB access classes (2)

2. Retrieving data

Now you want to retrieve the ZDC calibration object to calibrate, say, run 50 from our local DB storage

First: Create the AliRunDataStorage instance (AliRunDataOrganizedFile)

AliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);

Second: if necessary, specify some selection criteria using an AliSelectionMetaData object and the AliRunDataStorage::Select method

AliSelectionMetaData smd(“ZDC/Calib/*”,1,100,2);dof->Select(smd);

Continue…

This means: For all objects named“ZDC/Calib“, and for all runs between 1

And 100, I want version 2



2. Retrieving data (continued)

Fourth: get the AliZDCCalibData object valid for run 50 with AliRunDataStorage::Get method:

Third: if you want to dump the retrieved object(s) into a local (AliRunData) file, call the AliRunDataStorage::RecordToFile option before getting the objects:

dof->RecordToFile(“$HOME/MyDataBase.root”);

AliZDCCalibData *zdcdata = AliRunDataStorage::Instance()->Get(“ZDC/calib/Pedestals”,50);

Most general way to call the active AliRunDataStorage instance



For further examples:

• See the note “AliRoot Framework for Access to the Offline Calibration and Alignment Database” distributed by email

• See the tutorial macro $ALICE_ROOT/macros/DBAccessTutorial.C

• Follow the session “Hands on the DB access classes” next Friday morning


Further comments

• About the AliRunDataStorage::Select method:

More than one selection criterion can be specified in cascade, e.g.:

// Select version 2 for all TPC objectsAliSelectionMetaData smd(“TPC/*”);smd.SetVersion(2);AliRunDataStorage::Instance()->Select(smd); // but version 3 for “TPC/Calib/Gain”smd.SetName(“TPC/Calib/Gain”);smd.SetVersion(3);AliRunDataStorage::Instance()->Select(smd); // and version 0 for “TPC/Calib/DriftVelocity” //for runs 10 to 20AliSelectionMetaData smd2(“TPC/Calib/DriftVelocity”,

10,20,0);AliRunDataStorage::Instance()->Select(smd2);

Note: if during retrieval a selection criterion is specified, but that particular version of the object is not found, then the closest preceding version is returned. Suggested behaviour: is that what we want?


Further comments (2)

• About the automatic versioning of the stored objects (with

AliRunDataOrganizedFile class):

I stress that the version of the stored object is incremented if its run range “overlaps“ the one of an already existing object.

Example: suppose I want to update the calibration object stored in DBFolder/.../Run1-10_v0.root.

After the update, I realize that the object is valid up to run 15.

Then I just need to set the new run range in the object‘s metadata and the object will be stored in the file DBFolder/.../Run1-15_v1.root.


Further comments (3)

• Finally, remember that the objects can be retrieved via the

AliRunLoader class, through the function GetObject(“objectName”).Example (user macro):

// create an AliRunLoader objectAliRunLoader *runLoader = new AliRunLoader()…// open some AliRunDataStorage instance, specify selection // criteria, etc etc……// get the objectTObject *obj = runLoader->GetObject(“objectName”);


Work in progress

• First implementation of AliRunDataStorage::TagForProduction method:

• With the AliRunDataOrganizedFile (and probably with AliRunDataAlien) access

classes the simplest solution may be to create symbolic links of the files and label them with a “production version”.

void TagForProduction(AliSelectionMetaData &smd, Int_t prodVersion)

The AliSelectionMetaData contain all the

information needed to identify the object(name, run range, version)

• We have been suggested to provide a means to “tag” a specific version of every objects, in order to ease their retrieval in case they should be used for global data reconstruction

“production” version that will label the

symbolic link to the DB object


Work in progress (2)

• Example of usage of TagForProduction: I want to tag the object in the

“organized” file DBFolder/ZDC/Calib/Pedestals/Run1-10_v3.root for production 0.

// open the AliRunDataOrganizedFile instanceAliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);// Create the SelectionMetaData object with the info needed // to identify the objectAliSelectionMetaData smd(“ZDC/Calib/Pedestals”,1,10,3)// Tag the file:dof->TagForProduction(smd,0)

• The following symbolic link is created in the same directory as the object:

DBFolder/…/Run1-10_Prod0.root → Run1-10_v3.root


Work in progress (3)

• I propose a method to prepare AliRunDataStorage to retrieve the data valid for production:

void AliRunDataStorage::SetStorageMode(int storageMode)

• If storageMode=1 (“Production mode”) then the retrieval is done in the standard

way for all files tagged for production.

Note: Discussion on this issue is under way within the core offline group (see Magali‘s talk).

alberto colla - cern alice off-line week 1 alberto colla alice off-line week cern, may 31, 2005...

Documents