apr. 8, 2002calibration database browser workshop1 database access using d0om h. greenlee...

Apr. 8, 2002 Calibration Database Browser Workshop 1

Database Access Using D0OM

H. Greenlee

Calibration Database Browser Workshop

Apr. 8, 2002


D0OM Components

• D0OM Object Model.

• Preprocessor and dictionary system.

• I/O Interface.– Non-database: d0Stream.

• Open/close/read/write.

– Database: d0StreamDB.• Query.

• Calibration Data Model.– Superset of D0OM object model.

• I/O back ends.– DSPACK/EVPACK.

– Direct Oracle (d0omORACLE).• C++ to database mapping dictionary.

– Client-Server Oracle (d0omCORBA).• C++ to database mapping dictionary.


D0OM I/O Classes


Calibration Architectures

CalibrationManager

d0omORACLE OracleDatabase

D0OMDictionary

MappingDictionary

CalibrationManager

LowLevelServer

MappingDictionary

D0OMClient

D0OMDictionary

HighLevel

D0OMServer

Direct Database Connection

Database Client-Server Architecture

CalibrationData

Model

CalibrationData

Model

OracleDatabase


Client Server Architecture

• Client and server use corba protocol to communicate.

• Client written in C++ (e.g part of d0reco).– Uses orbacus corba implementation.

– Dynamic corba (no compiled in knowledge of database schema).

• Only type any (no dynany).

• Server written entirely in python.– Uses omniorb corba implementation.

– Consists of high level part (schema independent) and low level part (schema-specific generated idl and python).

– Schema specific part generated using fnal product db_server_gen (script dbgen.py).


D0OM Object Model

• Atomic types.

• Strings.

• Arrays.

• Bare pointers.

• STL collections.

• D0OM base class: d0_Object.

• D0OM smart pointer: d0_Ref<T>.

• User-defined classes.


Calibration Data Model

• Database base class: d0_DBObject<K>.

• Query Classes.– d0_Query<T,C> and d0_Query_Base.

• Database Pointers.– d0_DynRef<T>.

– d0_RefTo<T,K>.

• Database Collections– d0_Query_Collection<T,C>.

– d0_RefBy<T,K,C>.

• Database key class.– Abstract class d0_DBKey.

– Concrete classes d0_DBKey_1<X>, d0_DBKey_2<X,Y>, etc.


Database Base Class

• Class d0_DBObject<K> contains database key.– Template parameter class K must fulfill

d0_DBKey interface requirement.

• Attribute name fields of key are filled automatically from dictionary when d0_DBObject<K> is constructed.


Database Pointer Classes

• The database pointer classes inherit D0OM smart pointer interface (d0_Ref<T>).

• The database pointer classes support true deferred I/O:– They can be initialized to point to an object

in a database without doing any I/O. (This was not true for dspack d0_Ref<T>.)

• d0_DynRef<T> represents a relationship based on a general query object, which can be specified at run-time.– Can not be stored in database.

• d0_RefTo<T,K> represents a static one-to-one relationship based on key class K.– Key is automatically converted to query.– Points to d0_DBObject<K> or

d0_RefTo<>/d0_RefBy<> (symmetric relationship).

– Can be stored in database.


Smart Pointer Class Diagram


Database Collections

• The database collection classes have interfaces similar to an STL collection.

• The database collection classes also support true deferred I/O.

• d0_Query_Collection<T,C> represents a run-time collection based on a general query (plural version of d0_DynRef<T>).

• d0_RefBy<T,K,C> represents a static one-to-many relationship based on key K (plural version of d0_RefTo<T,K>).


Query Classes

• Query classes d0_Query<T,C> and d0_Query_Base act as front end to query method d0StreamDB::query.

• d0_Query<T,C> returns collection of d0_Ref<T>’s when executed.

• d0_Query_Base returns a collection of d0_Ref_Any’s when executed.


Database Key Classes

• Abstract class d0_DBKey and subclasses.

• Unlike ordinary persistent data, database keys know the names of their database columns.– Keys can easily be turned into query

objects.– Keys can be constructed by hand, or

extracted from database base objects.


Public Indirect Pointers

• New public indirect pointer class d0om_Query_Indptr is not specific to any D0OM I/O mechanism (but requires d0StreamDB::query to be implemented by back end).


C++ to Database Mapping

• Goals:– Full support for D0OM object model,

including polymorphism (e.g. for writing event data).

– Flexible C++ to database mapping (e.g. for reading a human designed database).


Database Representation of C++ Objects.

• Ordinary (non-database) persistent objects (derived from d0_Object).

• Database persistent objects (derived from d0_DBObject<K>.

• Object id. & reference count columns are optional.

objid refcount K

M

n 0 x y K

M

D0OM Columns User Columns

objid refcount K K

M

n 0 k x y K

M



Database Representation (cont.)

• Ordinary pointers and d0_Ref<T>’s.

• Dynamic type column and type table are optional.

• Not use in D0.

MK n t KM

Ref

objid refcount K

M

n 1 x y K

M


Type id C++ TypeMt MyclassM



• Database smart pointer d0_RefTo<T,K>.

• Dynamic pointers are not polymorphic.

• Object id and reference count columns not used in D0.

RefMK k KM

objid refcount K K

M

n 1 k x y K

M




• STL Collection.

• Not used in D0.

CollMK n KM

objid element_number K

M

n 1 x y Kn 2 x y Kn 3 x y K

M




• Database collection d0_RefBy<T,K,C>.

• Object id and reference count not used in D0.

CollMK k KM

objid refcount K K

M

n 1 k x y Kn+1 1 k x y Kn+2 1 k x y K

M



External Database Dictionary

• D0omORACLE and d0omCORBA use an external python mapping dictionary.

• The following elements of C++ to database mapping can be controlled.– Class name Table name.

– Data member name Column name.

– Server-based foreign key relationships.

– Metadata table and column names (not currently used).

• External dictionary is input to db_server_gen.

• I believe that it is possible to generate database dictionary automatically (i.e. from oracle designer), but I don’t know how.


Generating server files using db_server_gen

• Need database dictionary files.– GENLIST.py (contains list of table modules)

– MASTER.py (list of directories)

– One python file for each database table.

– Python modules must be on $PYTHONPATH.

• Run script dbgen.py to generate low level server files:% setup db_server_gen

% dbgen.py –d0om

• Files generated:– Base layer idl files (one per table).

– Python server files.

– C++ header and implementation files.


Programming Using d0omORACLE/d0omCORBA

• Use stream factory to open stream.d0StreamFactory* factory = d0StreamFactory::locateStreamFactory();

d0StreamDB* stream = factory->make_d0StreamDB(name, “”, ios::in);

– Specific stream type is “ORACLE,” “CORBA,” or “” (empty string).

– For d0omORACLE, specify stream name as database connect string “username/password@database.” (password optional).

– For d0omCORBA, specify stream name as corba name of server (as registered with name server).


Programming Using d0omORACLE/d0omCORBA

• Linking.– Force load LoadORACLE.o or

LoadCORBA.o.

– Link with D0 libraries –ld0omORACLE or -ld0omCORBA.

– For product libraries use srt_d0 makefile fragments arch_spec_oracle.mk or arch_spec_corba_common.mk and arch_spec_orbacus.mk (automatic with ctbuild).


Running d0omCORBA programs

• Start the server.– Add dictionary files (GENLIST.py, etc.) and

generated python files to $PYTHONPATH.

– Setup products omniOrb, dcoracle, and sam_util (or sam).

– Specify database user name/password using env variables DB_SERVER_USERNAME and DB_SERER_PASSWORD.

– Start server:% DbListener.py –n=<name> -t=d0om –c=ServantFactory –m=<module>

• Running the client.– Setup fnal product sam_util or sam (makes

corba naming service available).

– Run client program.• Client can run on different node than server.

• Password is not required for client.


Documentation

• D0omCORBA and d0omORACLE– d0omCORBA/doc/database_access.ps

– d0omCORBA/doc/database_dictionary.ps.

• Calibration data model.– d0stream/doc/calibration_classes.ps.

• D0om generally.– d0om/d0c/d0om_user_guide.ps.

apr. 8, 2002calibration database browser workshop1 database access using d0om h. greenlee...

Documents