elizabeth gallas august 9, 2005 cd support for d0 database projects 1 elizabeth gallas fermilab...

Elizabeth Gallas August 9, 2005

CD Support for D0 Database Projects

1

CD Support forD0 Database Projects

Elizabeth GallasFermilab Computing Division

Fermilab CD Grid and Data Management Meeting

August 9, 2005 3-5pm



2

Outline

• Define “Support”• Specific “active” projects requesting “support”

– Notes• Projects in “maintenance” are not included here• Excluding work by myself (Elizabeth) ??

– Projects• Luminosity• Trigger• Calibration• Online

• Summary



3

Define “Support”• Hardware installation

• Hardware maintenance

• Backup recovery

• Fail over mechanisms

• Advice/oversight– synonyms

– loading data

– database triggers

– schema definition and changes

– cutting scripts

• Application Development where indicated



4

“Active” D0 Database Projects

• Luminosity Database

• Trigger Database

• Calibration Database(s)– Muon PDT (in development)

– Calorimeter (‘new’ in production)

• D0 Online databases

• Run Summary database (config db server)



5

D0 Luminosity Database: Overview • Purpose:

– Calculate and deliver a Luminosity Normalization

– Verify and certify data integrity for normalization purposes

– Improve error tracking and characterization

– Produce Operations Reports and Graphs

– Produce LBN Quality Reports

• Components:– 37 Oracle Tables, 2 Views on d0oflum1 in December 2004

temporarily on fcdfora3 until new machine arrives

– Web based Operations Report interface

– CORBA based database server for use by lmAccess• Get Luminosity normalization for a particular data set

– Interfaces for Data Quality Entry• Replacing “Quality by Run” with “Quality by LBN”



6

Luminosity: Effort/Support• Interfaces (lmAccess, Operations Reports, LBN Quality)

– developed primarily by D0 collaboration institutions/personnel with advice from Fermilab CD

– Interfaces reside in CD Database supported environment

• Database Application Infrastructure - Support from CD– New machine for Luminosity Database (production)

• now on cdfora3 moving to d0ora???

• backups, failover

– machine for Luminosity Database development • now on cdfrep02 moving to ?????? (old d0mino disks)

– New Oracle instance ‘d0oflum1’• synonyms pointing to Runs, Trigger … (done?)

• storage definitions (done) storage monitoring

• updates/patches for Oracle and other products

– Web server on ‘d0ora2’ (d0ofprd1) for Operations Reports

– Need help setting up SQUID (caching) server

– Database server (D0 Calibration style, not like SAM db server)



7

Trigger: Overview• Purpose:

– Critical to D0 online operations• Create, Modify, Store and Report -- all Run 2 D0 Trigger Configurations

• Generate precise configuration in ‘xml’ format: for online and simulation for a complex and evolving ‘device’

– Document – • Store/Retrieve configuration information from a Database for a User.

• TDB is the first place collaborators look to get an understanding of the D0 Trigger System

• Components:– Oracle: 40 Tables, 25 Views

– 3 Web based interfaces:• TDB Entry Interface (13 modules)

• TDB Report Interface (14 modules)

• TDB MISWEB Interface

– Trigger Db Server (like the SAM db servers, a customized db server)

– Many command line interfaces • including xmlgen, the trigger configuration generator



8

Trigger DB: Effort Profile• People who have worked on this project:

– Elizabeth Gallas, Vicky White, Jeremy Simmons, Matt Vranicar, Rich Wellner, John Weigand, Vijay Murthi, Steve White, Adam Lyon, Margherita Vittone, Eric Wicklund.

• People currently/recently working on the project:– Elizabeth Gallas

– Margherita Vittone – web Entry Interface (client using db server)

– Steve White – trigger db server and client interface to server

– Eric Wicklund (new) – web Report Interface (client using dcoracle)

Note: all individuals are from FNAL Computing Division.

Efforts to find non-FNAL D0 collaborators to contribute materially to the project have come to naught (up to this point).– Specialized nature of the application

– Requires ‘trigger’ and ‘database’ expertise

– Database expertise much more effective at making a contribution



9

Trigger: Eric WicklundReport Interface expert. Understands code and views. Has all

accounts and privileges.• Has completed (or nearly completed):

– tdb_objects (<1 day),

– tdb_terms (3 days to include L2pp *),

– tdb_L1dialogs - new,

– tdb_neoterms – new (<1 day)

– tdb_scripts (3 days to include L2pp *)

• Would be natural for him to write/fix the remaining modules:– tdb_tn (6 days),

– tdb_tl (6 days),

– tdb_tldependency (3 days)

– * tdb_l2pp – new (5 days)

– tdb_dg and tdb_ed – new (5 days)

Needs test links from Elizabeth (going into Project Plan)



10

Trigger: Margherita VittoneEntry Interface expert. Understands code. Has

accounts and privileges.• Working versions of:

– trigdb_status (4-5 days) + Steve White (2 days) – trigdb_l1dialogs (done)– trigdb_scripts (2-4 days) + 1 day for L2pp

• Working on:– trigdb_terms (2-4 days), 1 week (eliminate duplicates), 1 week L2pp– trigdb_triggerlists (1 week) implement text area mode

• Would be natural to complete needed changes to:– trigdb_objects (2-4 days), 3 days to eliminate duplicates– trigdb_triggernames (2-4 days)– trigdb_L2pp (1 week) + Steve White (2 days)

• Expert required to complete– trigdb_tltransform - Elizabeth



11

D0 Calibration Database(s): Overview • Purpose:

– Store and Report Calibration constants for various sub-detectors

• calibration,

• data reconstruction, and

• analysis

• Components:– ONLINE instance

– OFFLINE instance

– CORBA based database server used by reco processing farms on FNAL CD maintained machines

• Personnel:– FNAL PPD or D0 Collaborators from other institutions



12

Calibration: Applications • Status of 2 Applications in ‘development’:

– Calorimeter (Ursula) • used in production since Jan 05• experienced some hiccups, developing systematic checks

– Muon PDT (Linda Stutte)• now in development

– schema changes– db server development, first usage– writing data loading programs

– ‘config db server’ to be used by FPD (Luiz)• not an FPD calibration database but uses Run Summary DB

Help from Database experts like Anil and Steve at our D0 database meetings is critical to getting these types of projects quickly on track.



13

D0 Databases: Conclusion• Fermilab staff on experiments and in the Database Support Group(s)

play a key role in the success of these (and other) D0 database projects– But they need help and active participation from

• experienced developers and • database support (Lee’s group and Julie’s group)

• Luminosity: Significant components of some applications can be developed and maintained by University groups

• Trigger:– Specific requests are made for Eric Wicklund and Margherita Vittone’s time

on projects that they are configured to do

• Calibration:– Development of new schemas and servers is ongoing.

• Online Databases:– Support for backup to Vladimir, negotiations underway.

• Steve White’s expertise is in constant demand, though not much time for any one issue for any application.– Luminosity, Trigger, Calibration and Run Summary (Config db server).

elizabeth gallas august 9, 2005 cd support for d0 database projects 1 elizabeth gallas fermilab...

Documents