lhcb development
DESCRIPTION
LHCb Development. Glenn Patrick Rutherford Appleton Laboratory. b. B meson. d. b. d. LHCb - Reminder. 1.2M electronic channels Weight ~4,000 tonnes. Muon System. Tracking stations (inner and outer). Magnet. Calorimeters. RICH2. VELO. 20 m. RICH1. Anti-B meson. - PowerPoint PPT PresentationTRANSCRIPT
4th February 2004 GRIDPP9 1
LHCb Development
Glenn PatrickRutherford Appleton Laboratory
4th February 2004 GRIDPP9 2
LHCb - Reminder
RICH1RICH2
Calorimeters
Muon System
VELO
Magnet
Tracking stations(inner and outer)
20 m
1.2M electronic channelsWeight ~4,000 tonnes
b
d
B meson
b
dAnti-B meson
4th February 2004 GRIDPP9 3
LHCb GridPP Development
LHCb development has been taking place on three fronts: MC Production Control
and Monitoring Gennady Kuznetsov (RAL)
Data ManagementCarmine Cioffi (Oxford) Karl Harrison (Cambridge)
GANGA Alexander Soroko (Oxford)Karl Harrison (Cambridge)
All developed in tandem with LHCb Data Challenges
4th February 2004 GRIDPP9 4
Data Challenge DC03
VELO TT
RICH2
RICH1
65M events processed.
Distributed over 19 different centres.
Averaged 830,000 events/day.
Equivalent to 2,300 × 1.5GHz
computers.
34% processed in UK at 7 different institutes.
All data written to CERN.
“Physics” Data Challenge.Used to redesign and optimise detector …
4th February 2004 GRIDPP9 5
The LHCb Detector
Changes were made formaterial reduction andL1 trigger improvement
Reduced number of layers for M1 (4 2) Reduced number of tracking stations behind the magnet (4 3)
No tracking chambers in the magnetNo B field shielding plate
Full Si stationReoptimized RICH-1 design
Reduced number of VELO stations (25 21)
4th February 2004 GRIDPP9 6
“Detector” TDRs completed
Only Computing TDR remains
4th February 2004 GRIDPP9 7
Data Challenge 2004“Computing” Data Challenge.April – June 2004
Produce 10 × more events.
At least 50% to be done via LCG.
Store data at nearest Tier-1(i.e. RAL for UK institutes)
Try out distributed analysis.Test computing model and write computing TDR.
Require stable LCG2 release with SRM interfaced to RAL DataStore
4th February 2004 GRIDPP9 8
DC04: UK Tier-2 Centres
NorthGridDaresbury, Lancaster, Liverpool,Manchester, Sheffield
SouthGridBirmingham, Bristol, Cambridge,Oxford, RAL PPD
ScotGridDurham, Edinburgh, Glasgow
LondonGridBrunel, Imperial, QMUL, RHUL, UCL
1101100011
4th February 2004 GRIDPP9 9
DIRAC Architecture
Information Service
Authentication
Authorisation
Auditing
Grid Monitoring
Workload Management
Metadata Catalogue
File Catalogue
Data Management
Computing Element
Storage Element
Job Provenance
Package Manager
User Interface
API
Accounting
DIRAC components
Other project components:AliEn, LCG, …
Resources:LCG, LHCb production sites
4th February 2004 GRIDPP9 10
MC Control StatusGennady Kuznetsov
Control toolkit breaking down production workflow into components – modules, steps.
To be deployed in DC04. SUCCESS!
DIRACDistributedInfrastructure withRemoteAgentControl
4th February 2004 GRIDPP9 11
Bookkeeping dataMonitoring info
Get jobs
Site ASite A
Site BSite B
Site CSite C
Site DSite D
Agent
Production service
Monitoring serviceBookkeeping service
AgentAgent
Agent
DIRAC v1.0 Original scheme
“Pull” ratherthan “Push”
4th February 2004 GRIDPP9 12
Components – MC Control
Module
Module
Module
Module
Module
Module
Module
Step
Step
Step StepStep
Step Step
Step
WorkflowJob
Job Job Job
Job Job
JobJob
Job Job Job
Job Job
Job
Job
Job Job Job
Job Job
Job
ProductionLevels of usage:1. Module – Programmer2. Step – Production
Manager3. Workflow –
User/Production manager
Module is the basic component
of the architecture
Each step generates job as a Python program.
This structure allow the Production Manager to construct any algorithm as a combination of modules.
Gennady Kuznetsov
4th February 2004 GRIDPP9 13
Module Editor
Python code of single module. Can
be many classes.Module
variables.
Description
Module Name Stored as XML file
Gennady Kuznetsov
4th February 2004 GRIDPP9 14
Step EditorStep Name
Description
Definitions of Modules
Instances of Modules
Variables of currently selected instance
Selected instance
Stored as XML file, where all modules are embedded
Gennady Kuznetsov
Step variables.
4th February 2004 GRIDPP9 15
Workflow Editor
Gennady Kuznetsov
Workflow Name
Description
Step Definitions
Step Instances
Variables of currently
selected Step Instance
Selected Step Instance
Workflow Variables.
Stored as XML file
4th February 2004 GRIDPP9 16
Job Splitting
Gennady Kuznetsov
Step
Step StepStep
Step Step
Step
Workflow DefinitionJob
Job Job Job
Job Job
JobJob
Job Job Job
Job Job
Job
Job
Job Job Job
Job Job
Job
Production
Python List
The input value for the job splitting is a Python list object.
Every single (top level) element of this list applies to the Workflow Definition and propagates through the code and generates a single element of the production (one or several jobs).
4th February 2004 GRIDPP9 17
Future: Production Console
Once an agent has received a workflow, the Production Manager has no control over any function in a remote centre.
Local Manager must perform all of the configurations and interventions at individual site.
Develop ”Production Console” which will provide extensive control and monitoring functions for the Production Manager.
Monitor and configure remote agents.
Data replication control.
Intrusive system – need to address Grid security mechanisms and provide robust environment.
4th February 2004 GRIDPP9 18
DIRAC v1.0 Architecture
Application packager
Application packager
Workfloweditor
Workfloweditor
Production editor
Production editor
Production manager
Production DBProduction DB
Production preparation
EditInstantiateworkflow
Createapplication
tar file
Central Services
MonitoringDB
MonitoringDB
BookkeepingDB
BookkeepingDB
Production resources
Agent AAgent ASite A
Agent nAgent nSite n
…
JobXMLJobXML
JobstatusJob
status
MetaXMLMetaXML
Castor MSSCERN
Castor MSSCERN
Datasetreplica
Datasetreplica
Agent BAgent BSite B
Central Storage
Jobrequest
Jobrequest
Bookkeeping Service
Bookkeeping Service
Monitoring Service
Monitoring Service
Production Service
Production Service
Application packager
Application packager
Workfloweditor
Workfloweditor
Production editor
Production editor
Production manager
Production DBProduction DB
Production preparation
EditInstantiateworkflow
Createapplication
tar file
Central Services
MonitoringDB
MonitoringDB
BookkeepingDB
BookkeepingDB
Production resources
Agent AAgent ASite A
Agent nAgent nSite n
…
JobXMLJobXML
JobstatusJob
status
MetaXMLMetaXML
Castor MSSCERN
Castor MSSCERN
Datasetreplica
Datasetreplica
Agent BAgent BSite B
Central Storage
Jobrequest
Jobrequest
Bookkeeping Service
Bookkeeping Service
Monitoring Service
Monitoring Service
Production Service
Production ServiceProduction Manager
4th February 2004 GRIDPP9 19
DIRAC v2.0 WMS Architecture
Job Receiver
Job Receiver
Job DBJob DB
Optimizer 1Optimizer 1Optimizer 1Optimizer 1
Optimizer 1Optimizer 1
MatchMakerMatchMakerJo
b qu
eue
Agent 1Agent 1
Agent 2Agent 2
Agent 3Agent 3
LCG CELCG CE
LCG WMSLCG WMS
DIRAC CEDIRAC CE
DIRAC Workload Management
Productionservice
Productionservice
GANGAGANGA
Commandline UI
Commandline UI
Computing resources
Job Receiver
Job Receiver
Job DBJob DB
Optimizer 1Optimizer 1Optimizer 1Optimizer 1
Optimizer 1Optimizer 1
MatchMakerMatchMakerJo
b qu
eue
Agent 1Agent 1
Agent 2Agent 2
Agent 3Agent 3
LCG CELCG CE
LCG WMSLCG WMS
DIRAC CEDIRAC CE
DIRAC Workload Management
Productionservice
Productionservice
GANGAGANGA
Commandline UI
Commandline UI
Computing resources
ProductionService
Also data stored remotely
Based on central queue service
4th February 2004 GRIDPP9 20
Data Management Status
Carmine Cioffi
File catalogue browser for POOL
Integration of POOL persistency framework into GAUDI new EventSelector interface.
SUCCESS!
4th February 2004 GRIDPP9 21
List of LFNs
Tabs for LFN / PFN
mode selection
List of PFNs associated to the
LFN selected from the list of
LFNs on the left sub-panel
Read the next and previous bunch of
files from the catalog
Write mode selection
Import the fragment of a catalog
Reload the catalog
Shows the metadata schema, with the
possibility to change it
List all the metadata value of the catalog List the files
selected
Search text bar
Filter text bar.
Main Panel, LFN Mode Browsing
POOL file catalogue providesLFN & PFN association.
Browser allows user to interact with catalogue via GUI.
Can save list of LFNsfor job sandbox
4th February 2004 GRIDPP9 22
Main Panel, PFN Mode Browsing
Sub menu with three operations to be done on the file
selected.
In PFN mode, the files are browsed in the same way as Windows Explorer. The folders are shown on the left sub-panel and the value of the folder on the right sub-panel.
Write mode buttonopens WrFCBrowserframe allowing user to write to the catalogue…
4th February 2004 GRIDPP9 23
Register a PFN
Add a PFN replica
Delete a PFN
Add LFN
Remove a LFN
Add metadata
value
RollbackCommit
Show the action performed
Write Mode Panel
4th February 2004 GRIDPP9 24
PFN register frame
Frame to show and change the metadata schema of the catalog
This frame allows setting of the
metadata value
4th February 2004 GRIDPP9 25
This frame shows the
metadata value of the PFN Myfile
Shows the list of the files selected
This frame shows the attribute value of the
PFN
4th February 2004 GRIDPP9 26
Benefit from investment in LCGRetire parts of Gaudi reduce maintenance.
Designed and implemented a new interface for the LHCb EventSelector. Criteria:
One or more “datasets” (e.g. list of runs, list of files matching a given criteria).
One or more “EventTagCollections” with extra selection based on Tag values.
One or more physical files.
Result of an event selection is a virtual list of event pointers.
GAUDI/POOL Integration
4th February 2004 GRIDPP9 27
Physicist’s View of Event Data
GaudiBookkeeping
DatasetEvent 1Event 2…
Event 3
DatasetEvent 1Event 2…
Event 3
FileEvent 1Event 2…
Event N
FilesRAW2-1/1/2008RAW3-22/9/2007RAW4-2/2/2008…
DatasetEvent 1Event 2…
Event 3
DatasetEvent 1Event 2…
Event 3
Event tag collctnTag 1 5 0.3Tag 2 2 1.2…
Tag M 8 3.1
Collection SetB -> ππ Candidates (Phy)B -> J/Ψ (μ+ μ-) Candidates…
4th February 2004 GRIDPP9 28
Future: Data to Metadata
File catalogue holds only a minimal amount of metadata.
LHCb deploys a separate “bookkeeping” database service to store the metadata for datasets and event collections.
Based on central ORACLE server at CERN with query service through XML-RPC interface.
Not scaleable, particularly for Grid, and completely new metadata solution required.
ARDA based system will be investigated.
Vital that this is development is optimised for LHCb and synchronised with data challenges.
Corresponds to ARDA Job Provenance DBand Metadata Catalogue
4th February 2004 GRIDPP9 29
DIRAC
Metadata: Data Production
Information Flow
Job.xml
•Build newconfiguration•Selection ofDefaults
Productiondone
Prod.Mgr
Configuration
Bookkeeping
Data Production
Production Jobs
File Catalogue
4th February 2004 GRIDPP9 30
Metadata: Data Analysis
User Job
Information Flow
Job.opts
ModifyDefaults
User
Select input data
Pick-up defaultconfiguration
Bookkeeping
Configuration
File Catalogue
DIRAC
4th February 2004 GRIDPP9 31
LHCb GANGA StatusAlexander Soroko, Karl Harrison
User Grid Interface.First prototype released in April 2003.To be deployed for LHCb 2004 Data
Challenge.SUCCESS!
LHCb
ATLAS
BaBar+ Alvin TanJanusz Martyniak
4th February 2004 GRIDPP9 32
GANGA for LHCb
GANGA will allow LHCb user to performstandard analysis tasks:
Data queries.Configuration of jobs, defining the job
splitting/merging strategy.Submitting jobs to the chosen Grid resources.Following the progress of jobs.Retrieval of job output.Job bookkeeping.
4th February 2004 GRIDPP9 33
GANGA User Interface
Database of Standard Job Options
Job Options Editor
Strategy Database (Splitting scripts)
Strategy SelectionData Selection
(Input/Output Files)
Job Requirements
(LSF Resources, etc)
Job Factory (Job Registry Class)
Ganga Job object Ganga Job object Ganga Job object Ganga Job object Ganga Job object
Local Client
Grid/Batch System Gatekeeper
Submit job
Send job output
Worker nodes
Get
jo
b o
utp
ut
Sen
d
Job script
JDL file
Job Options file
Get
Mo
nit
ori
ng
In
fo
Storage Element
File Transfer
4th February 2004 GRIDPP9 34
Software Bus User has access to
functionality of Ganga components through GUI and CLI, layered one over the other above a Software Bus
Software Bus itself is a Ganga component implemented in Python
Components used by Ganga fall into 3 categories: Ganga components of
general applicability or Core Components (to right in diagram)
Ganga components providing specialised functionality (to left in diagram)
External components (at bottom in diagram)
User has access to functionality of Ganga components through GUI and CLI, layered one over the other above a Software Bus
Software Bus itself is a Ganga component implemented in Python
Components used by Ganga fall into 3 categories: Ganga components of
general applicability or Core Components (to right in diagram)
Ganga components providing specialised functionality (to left in diagram)
External components (at bottom in diagram)
Job Definition
Job Registry
Job Handling
File Transfer
Python Native
Softw
are Bus
CLI
GUI
Pyth
on
R
oo
t
Gau
di
Pyth
on
PyC
MT
PyA
MI
Py
Mag
da
BaBar Job Definition and
Splitting
Gaudi/Athena Job Options
Editor
Gaudi/Athena Job Definition
4th February 2004 GRIDPP9 35
GUIs Galore
4th February 2004 GRIDPP9 36
DIRAC WMS Architecture
Job Receiver
Job Receiver
Job DBJob DB
Optimizer 1Optimizer 1Optimizer 1Optimizer 1
Optimizer 1Optimizer 1
MatchMakerMatchMakerJo
b qu
eue
Agent 1Agent 1
Agent 2Agent 2
Agent 3Agent 3
LCG CELCG CE
LCG WMSLCG WMS
DIRAC CEDIRAC CE
DIRAC Workload Management
Productionservice
Productionservice
GANGAGANGA
Commandline UI
Commandline UI
Computing resources
Job Receiver
Job Receiver
Job DBJob DB
Optimizer 1Optimizer 1Optimizer 1Optimizer 1
Optimizer 1Optimizer 1
MatchMakerMatchMakerJo
b qu
eue
Agent 1Agent 1
Agent 2Agent 2
Agent 3Agent 3
LCG CELCG CE
LCG WMSLCG WMS
DIRAC CEDIRAC CE
DIRAC Workload Management
Productionservice
Productionservice
GANGAGANGA
Commandline UI
Commandline UI
Computing resources
GANGA
4th February 2004 GRIDPP9 37
Future Plans
Database ofStandard Job Options
Job-Options Editor
Job-Options Template
Job-OptionsKnowledge Base
Dataset
Dataset Catalogue
Dataset Selection
Job Factory(Machinery for Generating XML Descriptions of Multiple Jobs)
StrategySelection
Job Collection(XML Description)
User Requirements
Database ofJob Requirements
Derived Requirements
Job Requirements
Strategy Database(Splitter Algorithms)
DispatcherScheduler Proxy
Scheduler Service
Remote-Client SchedulerGrid/ Batch-System
SchedulerAgent
(Runs/Validates Job)
Software CacheComponent
Cache
Software/Component Server
Remote Client
Local Client
Execution node
NorduGridLocalDIALDIRACOther
JDL, Classads,
LSF Resources, etc
LSFPBSEDGUSG
Refactorisation of Ganga, with submission on remote client
Refactorisation of Ganga, with submission on remote client
Motivation Ease
integration of external components
Facilitate multi-person, distributed development
Increase customizability/flexibility
Permit GANGA components to be used externally more simple
Motivation Ease
integration of external components
Facilitate multi-person, distributed development
Increase customizability/flexibility
Permit GANGA components to be used externally more simple
2nd GANGA prototype ~ April 2004
4th February 2004 GRIDPP9 38
Future: GANGADevelop into generic front-end capable of submitting a range of applications to the Grid. Requires central core and modular structure (started with version 2 re-factorisation) to allow new frameworks to be plugged in.
Enable GANGA to be used in complex analysis environment over many years for many users. Hierarchical structure, import/export facility, schema evolution, etc.
Interact with multiple Grids (e.g. LCG, NorduGrid, EGEE…). Needs to keep pace with development of Grid services. Synchronise with ARDA developments.
Interactive analysis? ROOT, PROOF