the hera-b detector the database problem the architecture the berkeley-db dbms
DESCRIPTION
The HERA-B database services detector configuration, calibration, alignment, slow control, data classification. A. Amorim, Vasco Amaral, Umberto Marconi, Tome Pessegueiro, Stefan Steinbeck, Antonio Tome, Vicenzo Vagnoni and Helmut Wolters. The HERA-B detector The database problem - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/1.jpg)
A. Amorim
1
The HERA-B database servicesdetector configuration, calibration,
alignment, slow control, data classification
• The HERA-B detector• The database problem• The Architecture• The Berkeley-DB DBMS• The client/server integration• The domains and solutions• Conclusions and Outlook
A. Amorim, Vasco Amaral, Umberto Marconi, Tome Pessegueiro, Stefan Steinbeck, Antonio Tome, Vicenzo Vagnoni and Helmut Wolters
![Page 2: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/2.jpg)
A. Amorim
2
HERA-B Experiment B/B-tagging B/B-tagging
B0/B0 J/KS B0/B0 J/KS
Vertex Detector Si strip 12 m resolution
RICH (/K)multianode PMT
TRD (eh)straw tubes+thin fibers
ECAL(+eh)W/Pb scintillator
shashlik
MUON (h)tube,pad and gaspixel chambers
Tracking: - ITR(<20cm): MSGC-GEM - OTR(>20cm): 5+10mm drift cells
Magnet: 2 Tm
C4F10
HiPt triggerpad/gas pixel
![Page 3: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/3.jpg)
A. Amorim
3
Magnet
Muon 1,3,4
SVD(not visible)
The HERA-B detector
ITR- OTR chambers
TRDOTR
ECALRICH
![Page 4: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/4.jpg)
A. Amorim
4
The main challenge: Selecting
![Page 5: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/5.jpg)
A. Amorim
5
How do we select them?
Time scale
Input rate
10s
10MHz
5ms
50 kHz
200ms 4 s
500 Hz 50 Hz 20 Hz
Pretrigger: ECAL, System, p T padsL1: e/“Tracking” in4 SL, p T cut, mass cut
Pretrigger: ECAL, System, p T padsL1: e/“Tracking” in4 SL, p T cut, mass cut
1/200
L2: + drift times, magnettraversal, vertexing
L2: + drift times, magnettraversal, vertexing
L3: full track & vertexfit, +SVD tracks, p. id
L3: full track & vertexfit, +SVD tracks, p. id1/100
L4: + full reconstruction,physics selection
L4: + full reconstruction,physics selection
TAPETAPE
1/2.51/10
![Page 6: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/6.jpg)
A. Amorim
6
HERA-B DAQ
L2-farm: 240 PC’s L4-farm 200 PC’s
1000 SHARC (DSP)
Detector Front End Electronics
FCS
DSP SWITCH DSP SWITCHEvent Control
TriggerPC
TriggerPC
TriggerPC
INTERNET SWITCH
4LTPC
4LTPC
LoggerPC
SLT/TLT
4LT
![Page 7: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/7.jpg)
A. Amorim
7
The HERA-B database problem
To provide persistence services (including online-offline replication) to:
• Detector configuration
• Common accepted schema
• Calibration and alignment • Distributing information to the reconstruction and trigger farms
• Associate each event with the corresponding database information
•Slow control
• Manage updates without data redundancy
•Data set and event classification
•Online Bookkeeping
• Detector Configuration
• Calibration and Alignment
• Slow Control
• Data Set and Event Classification
• Online Bookkeeping
![Page 8: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/8.jpg)
A. Amorim
8
Characterizing the context
Inter-processdata-flow rate
Complexity
Persistent Data-Volume
ARTE in memory structures:· Accessed very often by 1 process· Transient data limited to memory· No versioning, high complexity
Event Stream:· Accessed 2-3 times by 1 process· Large integrated data volume· Relatively simple structure
Database Services:· Accessed every data-set or slow control
update by Several hundred processes· Persistent data always available for retrieval· Time and version, high complexity
Analysis
20 Terabytes /year
Gigabyte/update
Large number of classesMerging data and updates
Complex queries
![Page 9: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/9.jpg)
A. Amorim
9
Querying on time intervals
one can select objects by the values of their attributes
Most of our requests want to query on time or version
exception: Event Tag Database: select on (Part Type, E, Pt, etc).
Query on time => Object(Time) or Object(t1,t2)
Our simple database layer on top of Berkeley DB provides that
otherwise one has to specialize the DBMS
( example the conditions database of BaBar and R&D45)t
![Page 10: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/10.jpg)
A. Amorim
10
Keys, objects and client/server
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
client/server at the SDB level +RPM -> an UDP based communication package.
/PM/ Descrip. field1 ; field 2; ...
.2
.5-.1
56892
...Db: /RICH/HV/
Db: /RICH/HV/
versions
Key= name+ version
Machine independent blub of DATA
![Page 11: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/11.jpg)
A. Amorim
11
The Berkeley DB
See http://www.sleepycat.com/
• Embedded transactional store with: logging, locking, commit and roll back, disaster recovery.
•Intended for: high-concurrency read-write workloads, transactions and recoverability. • Cursors to speed access from many clients..
• Open Source policy The license is free for non-commercial purposes - rather nice support
•No client/server support is provided
![Page 12: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/12.jpg)
A. Amorim
12
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control Interface
Slow Control interface Client/server
Metadata Object Data Object Update
time
Pmt1000Pmt1003Pmt2000
Update
1 ...1.5 ...2 ...
1.2 ...
1.6 ...2.3 ...
slctemp
nfields : intcolumsize : intfieldtype : int*xbuf : void *ids : int *columdata : void *
LocCopy2Table( )LocCopyTable( )LocMergeSlcTable( )
MIZZItable
SlcDatabase
x : Tablesdbname : char *
SlcBookDatabase( )
db
1..1
.sch tupd.dat
tdat
SlcTable
nused : int = 0nmax : intcarray : char **tabledes : char *tabledat : char *tableupd : char *maxnupd : intslctemp : slctempdat *nchardef : int
SlcBookTable( )SlcUnbookTable( )SlcGetDatabase( )SlcOpenTables( )SlcCloseTables( )SlcAdd2Table( )SlcFindInTable( )SlcGetDescTable( )SlcPutDescTable( )SlcStoreUpdTable( )SlcStoreDataTable( )SlcMemTable( )SlcTableValues( )SlcTableGetpt( )SlcTableGetSpt( )
1..1
(trough the tabledes name )
.upd
Optimized Queries
![Page 13: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/13.jpg)
A. Amorim
13
Associations to Events
Index Objects (referenced by events) Client/server
Active server interface
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control interface Client/server
Index Obj. Index Obj.
... ...
Index Obj.
...
Revision 0 - online
Index Obj.
Calibrating
1 - offline
Dynamic Associations
Index Obj. Created in active
Servers
![Page 14: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/14.jpg)
A. Amorim
14
The index API design
servers
transtabs
nend
nstartkey_server
name : char *vnew : int
key
n_servers : intmax_servers : intn_tables : intmax_tables : intn_transtabs : intmax_transtabs : int
add2servers( )add2tables( )add2transtabs( )marknotused( )cleannotused( )
tables
transtab
key_transtab
x : Tableskeyt : intmajor : intminor : intvnew : intline : intdbname : char *tablename : char *
key_table
name : char *major : intminor : intvnew : int
transtab == line
Index objects can associate also to transaction objects which are not data but are associated to sets of data objects that must be considered together.
Tools were also developed
![Page 15: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/15.jpg)
A. Amorim
15
The Parameter Distribution (cont.)
Calibration/Alignment Process Calibration/Alignment Process
C&A Manager
These are the first servers to startup
startup in second place startup in
third place
Db_Server
Db_Server
Db_Server
KeyDb_server
5: StoreTable(Table)
1: SUBSCRIBE(Table)
6: UPDATE(Table)
2: SUBSCRIBE(Table)
3: SUBSCRIBE(Table)
4: SUBSCRIBE(KeyTable)
7: UPDATE(KeyTable)
![Page 16: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/16.jpg)
A. Amorim
16
![Page 17: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/17.jpg)
A. Amorim
17
Basic n-n associations (LEDA)
LEDA - Object Manager (hash table implemented associations)
Key objects (referenced by events) Client/server
Active server interface
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control interface Client/server
Key key’
Key key’
•Associations are navigated with iterators•Using hash tables.•Keys as OID’s with the scope of classes. •Explicitly loaded or saved (as containers)
![Page 18: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/18.jpg)
A. Amorim
18
GUI for editing and drawing
From R&D: JAVA, TCL/TK, gtkReusing and extending widget.
Data hidden from TCL/TK
ROOTdatabaseBindingSocket:
Client/ Server
![Page 19: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/19.jpg)
A. Amorim
19
General Architecture
SDB/local thin layerMIZZI
C- formatter/client
Datastore ServerSlow Control interface
Key objects (referenced by events)
Replication
Tcl/TkOpen Source
SDB/client thin layer
ROOTOpen Source
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
LEDA - Object Manager(hash table implemented associations)
Berkeley DB / Sleepycat embedded database managerOpen Source
Database GUI Spreadsheet
Db GUIPlotting
Root ObjectPersistency
Client/server
Client/server
Socket
Active serverinterface
MySql, RDBMSOpen Source
Tag and Data Quality
Tag and D/Q replication
109 Evt./y
![Page 20: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/20.jpg)
A. Amorim
20
The Cache Server
Userclient
Userclient
Userclient
Cache
Cache
Db- server
Db- server
Cache
Memory
WAN TCP/IP gateway
![Page 21: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/21.jpg)
A. Amorim
21
The Replication Mechanism
DB server
ONLINE OFFLINEfirewall
DB server
Incremental
dump files
Offline DB server
imported
Send to tape Offline
DB server
![Page 22: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/22.jpg)
A. Amorim
22
DAQ configuration
Software Components
Logical Hardware
Physical General Hardware
Fast Control System
FLT
Slow Control Hardware
Subdetector FED
![Page 23: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/23.jpg)
A. Amorim
23
DAQ (Software
Components)
DAQ_ProcTemp
type : const char[64]name : char[64]path : char[64]exec : char[64]env : char[256]args : char[256]restart : unsigned int
(from DAQ_CONF)
Software Component:Run Configuration
Software ComponentSMC Configuration
Software ComponentProcess Templates
Software ComponentProcess Configuration Trees
ProcCfg
RunCfg
DAQ_RunCfg
name : char [64](from DAQ_CONF)
ProcCfg
SMCCfg
DAQ_SMCCfg
name : char [64]sst : unsigned intgsdt : char [256]sdt : char [256]
(from DAQ_CONF)
up
down
NodeGrp
ProcCfg
ProcTemp
ProcCfg
DAQ_ProcCfg
grname : char[64]domain : char[64]name : char[64]range : unsigned int [2]env : char[256]args : char[256]init : char[64]init_args : char[256]external : unsigned intcritical : unsigned intrestart : unsigned intsynch : unsigned intfini : char[64]fini_args : char[256]
(from DAQ_CONF)
DAQ_R_RunProc
DAQ_R_SMCCfg
DAQ_R_ProcCfg
IUdownIUup
NodeGrp
DAQ_NodeGrp
name : char[32]type : char[32]
(from DAQ_CONF)
DAQ_R_ProcNgrp
ProcTemp
Software Components
DAQ_R_ProcTemp
DAQ_R_UITemp
DAQ_R_TempNgrp
![Page 24: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/24.jpg)
A. Amorim
24
VDS databases
![Page 25: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/25.jpg)
A. Amorim
25
Maintaining the system
• A slow control process is permanently checking the state of the database servers.
• It issues alarms for the detector shift crew
• Tools to start and stop the dynamic configuration of database servers are to be used by a set of experts.
• The configuration and startup of the distributed database server system is performed using a special configuration database for this system.
![Page 26: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/26.jpg)
A. Amorim
26
ConclusionsONLINE:• Large number of Clients => Gigabytes per Update
•broadcast simultaneously to SLT• tree of cache database servers to the 4LT
• Correlates (dynamically) each event with the databases objects• 600 k SLC parameters using data and update objects
•parameter history is re-clustered on the database servers • The online database system has been successfully commissioned
OFFLINE:• Replication mechanism decouples online from offline
•also provides incremental backup of the data • TCP/IP gateways and proxies
• “data warehousing” for data-set classification -> MySQL • Relation to event tag under evaluation• Also providing persistency to ROOT objects
Using Open Source external packages has been extremely useful.
![Page 27: The HERA-B detector The database problem The Architecture The Berkeley-DB DBMS](https://reader033.vdocuments.us/reader033/viewer/2022051401/5681516e550346895dbf9ea8/html5/thumbnails/27.jpg)
A. Amorim
27
Future directions ...
CORBA
Persistent State ServicePSS
Berkeley DBPersistent State Service
PSS
MySQL
Open Communications Interface (OCI)
IIOP UDP/ based
Client Client Client
(ORBacus)Client
Client
One must have a plan even if it is a wrong one ...
FarmsWAN